2024-04-15T15:09:14.392 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-15T15:09:14.392 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136 --name leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} -- /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/orig.config.yaml 2024-04-15T15:09:14.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-15T15:09:14.529 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136 branch: wip-lusov-quiesce-xlock description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} email: null first_in_suite: false job_id: '7657136' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce-xlock ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow request sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce-xlock sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5929 sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 sleep_before_teardown: 0 suite: fs suite_branch: wip-lusov-quiesce-xlock suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFuSZ/DP/wyAzuGlxUJi84I4XzTKzK+ihI5+e8VoYfLqsqaJE6ZyFbAYK2v+KqUj7YOOSmmxA9V2zQuEE9dpV8= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB4AzTXlZerE2v7/Ddv2PX3441QCJcjmCtHgGL+/aNwq5dX8jKtNz9IMUa1Wui1rXYF+npugXtLCPABO/UijQRk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_quiesce teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-15_14:54:08 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-15T15:09:14.529 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa; will attempt to use it 2024-04-15T15:09:14.530 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks 2024-04-15T15:09:14.530 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-15T15:09:14.532 INFO:teuthology.task.internal:Checking packages... 2024-04-15T15:09:14.558 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14' 2024-04-15T15:09:14.558 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-15T15:09:14.558 INFO:teuthology.packaging:ref: None 2024-04-15T15:09:14.558 INFO:teuthology.packaging:tag: None 2024-04-15T15:09:14.558 INFO:teuthology.packaging:branch: wip-lusov-quiesce-xlock 2024-04-15T15:09:14.558 INFO:teuthology.packaging:sha1: 9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:09:14.558 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce-xlock 2024-04-15T15:09:14.698 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2861.g9bf1c4ba 2024-04-15T15:09:14.700 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-15T15:09:14.706 INFO:teuthology.task.internal:no buildpackages task found 2024-04-15T15:09:14.706 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-15T15:09:14.728 INFO:teuthology.task.internal:Saving configuration 2024-04-15T15:09:14.745 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-15T15:09:14.751 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:09:14.776 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 15:04:05.993206', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFuSZ/DP/wyAzuGlxUJi84I4XzTKzK+ihI5+e8VoYfLqsqaJE6ZyFbAYK2v+KqUj7YOOSmmxA9V2zQuEE9dpV8='} 2024-04-15T15:09:14.800 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 15:04:05.991228', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB4AzTXlZerE2v7/Ddv2PX3441QCJcjmCtHgGL+/aNwq5dX8jKtNz9IMUa1Wui1rXYF+npugXtLCPABO/UijQRk='} 2024-04-15T15:09:14.800 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-15T15:09:14.806 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-15T15:09:14.806 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-15T15:09:14.806 INFO:teuthology.run_tasks:Running task console_log... 2024-04-15T15:09:14.929 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3476cd60d0>, signals=[15]) 2024-04-15T15:09:14.929 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-15T15:09:14.936 INFO:teuthology.task.internal:Opening connections... 2024-04-15T15:09:14.936 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:09:14.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:09:15.029 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:09:15.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:09:15.119 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-15T15:09:15.126 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-04-15T15:09:15.143 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-04-15T15:09:15.143 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-15T15:09:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-15T15:09:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-04-15T15:09:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-15T15:09:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-15T15:09:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-04-15T15:09:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-04-15T15:09:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-15T15:09:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-15T15:09:15.200 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-04-15T15:09:15.200 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-15T15:09:15.200 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-15T15:09:15.200 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-15T15:09:15.200 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-15T15:09:15.200 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-15T15:09:15.201 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-15T15:09:15.225 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-04-15T15:09:15.242 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-04-15T15:09:15.242 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-04-15T15:09:15.299 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-04-15T15:09:15.299 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-15T15:09:15.300 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-15T15:09:15.301 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-04-15T15:09:15.325 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-15T15:09:15.334 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-15T15:09:15.358 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-15T15:09:15.358 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-04-15T15:09:15.360 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-04-15T15:09:15.376 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-15T15:09:15.382 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-15T15:09:15.382 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-04-15T15:09:15.416 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-04-15T15:09:15.590 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-15T15:09:15.647 INFO:teuthology.run_tasks:Running task kernel... 2024-04-15T15:09:15.662 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-15T15:09:15.662 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-15T15:09:15.662 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-15T15:09:15.662 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-15T15:09:15.663 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-15T15:09:15.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:09:15.679 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-15T15:09:15.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:09:15.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-15T15:09:15.863 DEBUG:teuthology.task.kernel:sha1 for client.1 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:09:15.864 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-15T15:09:15.864 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-04-15T15:09:15.882 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-361.el9.x86_64 2024-04-15T15:09:15.882 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:09:15.882 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-15T15:09:15.883 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum upgrade -y linux-firmware 2024-04-15T15:09:15.918 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:09:15.918 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-15T15:09:15.922 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-04-15T15:09:15.939 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-361.el9.x86_64 2024-04-15T15:09:15.939 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:09:15.939 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-15T15:09:15.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum upgrade -y linux-firmware 2024-04-15T15:09:18.955 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - AppStream 5.0 MB/s | 6.7 MB 00:01 2024-04-15T15:09:18.984 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - AppStream 5.0 MB/s | 6.7 MB 00:01 2024-04-15T15:09:20.364 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-04-15T15:09:20.625 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-04-15T15:09:22.165 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-15T15:09:23.920 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.0 MB 00:03 2024-04-15T15:09:27.021 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-04-15T15:09:31.171 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.2 MB 00:00 2024-04-15T15:09:32.631 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-15T15:09:37.623 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 21 MB 00:04 2024-04-15T15:09:42.664 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-15T15:09:45.144 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-04-15T15:09:45.150 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 1 Package 2024-04-15T15:09:45.151 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:09:45.151 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 375 M 2024-04-15T15:09:45.151 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-04-15T15:09:45.631 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 865 kB/s | 19 MB 00:22 2024-04-15T15:09:49.915 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.2 MB 00:00 2024-04-15T15:09:52.593 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-15T15:09:53.759 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-15T15:09:56.156 INFO:teuthology.orchestra.run.smithi187.stdout:linux-firmware-20240219-143.el9.noarch.rpm 34 MB/s | 375 MB 00:11 2024-04-15T15:09:56.157 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:09:56.157 INFO:teuthology.orchestra.run.smithi187.stdout:Total 34 MB/s | 375 MB 00:11 2024-04-15T15:09:56.158 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-04-15T15:09:56.257 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-04-15T15:09:56.257 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-04-15T15:09:58.687 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-04-15T15:09:58.687 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-04-15T15:09:58.817 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-04-15T15:10:01.302 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-15T15:10:01.317 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-15T15:10:01.317 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-15T15:10:01.317 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-15T15:10:01.317 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-04-15T15:10:01.317 INFO:teuthology.orchestra.run.smithi150.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-15T15:10:01.317 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:10:01.318 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-15T15:10:01.318 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-15T15:10:01.318 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 1 Package 2024-04-15T15:10:01.318 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:10:01.318 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 375 M 2024-04-15T15:10:01.318 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-15T15:10:04.902 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-04-15T15:10:05.183 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-15T15:10:05.275 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:10:06.666 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:10:06.667 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-15T15:10:07.068 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:10:07.068 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:10:07.068 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-04-15T15:10:07.068 INFO:teuthology.orchestra.run.smithi187.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-15T15:10:07.068 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:10:07.068 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-04-15T15:10:07.371 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.1... 2024-04-15T15:10:07.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:10:07.518 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-15T15:10:07.519 DEBUG:teuthology.orchestra.run.smithi187:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-15T15:10:07.521 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi187.front.sepia.ceph.com to complete... 2024-04-15T15:10:23.384 INFO:teuthology.orchestra.run.smithi150.stdout:linux-firmware-20240219-143.el9.noarch.rpm 17 MB/s | 375 MB 00:22 2024-04-15T15:10:23.385 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:10:23.385 INFO:teuthology.orchestra.run.smithi150.stdout:Total 17 MB/s | 375 MB 00:22 2024-04-15T15:10:23.385 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-15T15:10:23.489 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-15T15:10:23.489 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-15T15:10:25.955 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-15T15:10:25.955 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-15T15:10:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-15T15:10:32.470 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-15T15:10:32.558 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:10:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:10:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-15T15:10:33.956 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-15T15:10:33.957 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:10:33.957 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-04-15T15:10:33.957 INFO:teuthology.orchestra.run.smithi150.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-15T15:10:33.957 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:10:33.957 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-15T15:10:34.246 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-15T15:10:34.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-15T15:10:34.397 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-15T15:10:34.398 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-15T15:10:34.401 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi150.front.sepia.ceph.com to complete... 2024-04-15T15:10:52.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15 15:10:52 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-15T15:10:52.980 INFO:teuthology.orchestra.run.smithi150.stderr:FINISHED --2024-04-15 15:10:52-- 2024-04-15T15:10:52.980 INFO:teuthology.orchestra.run.smithi150.stderr:Total wall clock time: 19s 2024-04-15T15:10:52.980 INFO:teuthology.orchestra.run.smithi150.stderr:Downloaded: 1 files, 341M in 18s (18.8 MB/s) 2024-04-15T15:10:52.983 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-15T15:10:52.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-15T15:10:55.034 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying... ######################################## 2024-04-15T15:10:55.090 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing... ######################################## 2024-04-15T15:10:55.170 INFO:teuthology.orchestra.run.smithi150.stdout:Updating / installing... 2024-04-15T15:11:03.143 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-15T15:11:20.100 INFO:teuthology.orchestra.run.smithi150.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-15T15:11:43.076 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-15T15:11:43.076 DEBUG:teuthology.orchestra.run.smithi150:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-15T15:11:43.146 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:11:43.146 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:11:43.146 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:11:43.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-15T15:11:43.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-15T15:11:43.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-15T15:11:43.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-15T15:11:43.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-15T15:11:43.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-15T15:11:43.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-15T15:11:43.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-15T15:11:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-15T15:11:43.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-15T15:11:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-15T15:11:43.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-15T15:11:43.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-15T15:11:43.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-15T15:11:43.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-15T15:11:43.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-15T15:11:43.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-15T15:11:43.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-15T15:11:43.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-15T15:11:43.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-15T15:11:43.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-15T15:11:43.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-15T15:11:43.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-15T15:11:43.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-15T15:11:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-15T15:11:43.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-15T15:11:43.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-15T15:11:43.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-15T15:11:43.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-15T15:11:43.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-15T15:11:43.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-15T15:11:43.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-15T15:11:43.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-15T15:11:43.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-15T15:11:43.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-15T15:11:43.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-15T15:11:43.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-15T15:11:43.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-15T15:11:43.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-15T15:11:43.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-15T15:11:43.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-15T15:11:43.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-15T15:11:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-15T15:11:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-15T15:11:43.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-15T15:11:43.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-15T15:11:43.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-15T15:11:43.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-15T15:11:43.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-15T15:11:43.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-15T15:11:43.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-15T15:11:43.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-15T15:11:43.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-15T15:11:43.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-15T15:11:43.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-15T15:11:43.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-15T15:11:43.201 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-15T15:11:43.202 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-15T15:11:43.203 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-15T15:11:43.204 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-15T15:11:43.205 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-15T15:11:43.206 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-15T15:11:43.207 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-15T15:11:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-15T15:11:43.209 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-15T15:11:43.210 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-15T15:11:43.211 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-15T15:11:43.212 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-15T15:11:43.213 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-15T15:11:43.214 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-15T15:11:43.215 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-15T15:11:43.216 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-15T15:11:43.217 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-15T15:11:43.218 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-15T15:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-15T15:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-15T15:11:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-15T15:11:43.222 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-15T15:11:43.223 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-15T15:11:43.224 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-15T15:11:43.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-15T15:11:43.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-15T15:11:43.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-15T15:11:43.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-15T15:11:43.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-15T15:11:43.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-15T15:11:43.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-15T15:11:43.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-15T15:11:43.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-15T15:11:43.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-15T15:11:43.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-15T15:11:43.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-15T15:11:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-15T15:11:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-15T15:11:43.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-15T15:11:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-15T15:11:43.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-15T15:11:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-15T15:11:43.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-15T15:11:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-15T15:11:43.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-15T15:11:43.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-15T15:11:43.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-15T15:11:43.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-15T15:11:43.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-15T15:11:43.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-15T15:11:43.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-15T15:11:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-15T15:11:43.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-15T15:11:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-15T15:11:43.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-15T15:11:43.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-15T15:11:43.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-15T15:11:43.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-15T15:11:43.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-15T15:11:43.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-15T15:11:43.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-15T15:11:43.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-15T15:11:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-15T15:11:43.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-15T15:11:43.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-15T15:11:43.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-15T15:11:43.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-15T15:11:43.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-15T15:11:43.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-15T15:11:43.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-15T15:11:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-15T15:11:43.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-15T15:11:43.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-15T15:11:43.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-15T15:11:43.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-15T15:11:43.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-15T15:11:43.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-15T15:11:43.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-15T15:11:43.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-15T15:11:43.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-15T15:11:43.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-15T15:11:43.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-15T15:11:43.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-15T15:11:43.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-15T15:11:43.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-15T15:11:43.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-15T15:11:43.287 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-15T15:11:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-15T15:11:43.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-15T15:11:43.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-15T15:11:43.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-15T15:11:43.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-15T15:11:43.297 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:11:43.297 DEBUG:teuthology.orchestra.run.smithi150:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-15T15:11:43.319 INFO:teuthology.orchestra.run.smithi150.stdout:postinstall scriptlet (using /bin/sh): 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout:for file in vmlinuz System.map config; do 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout: fi 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout:done 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:11:43.320 INFO:teuthology.orchestra.run.smithi150.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:11:43.321 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-04-15T15:11:43.321 INFO:teuthology.orchestra.run.smithi150.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-15T15:11:43.321 INFO:teuthology.orchestra.run.smithi150.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:11:43.321 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-04-15T15:11:43.321 INFO:teuthology.orchestra.run.smithi150.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-15T15:11:43.321 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-15T15:11:43.321 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:11:43.321 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-04-15T15:11:43.322 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -qi grub2-tools 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:Name : grub2-tools 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:Epoch : 1 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:Version : 2.06 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:Release : 68.el9 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: x86_64 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:Group : Unspecified 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:Size : 8264254 2024-04-15T15:11:43.397 INFO:teuthology.orchestra.run.smithi150.stdout:License : GPLv3+ 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:Packager : builder@centos.org 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:Vendor : CentOS 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:Summary : Support tools for GRUB. 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:Description : 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-15T15:11:43.398 INFO:teuthology.orchestra.run.smithi150.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-15T15:11:43.399 INFO:teuthology.orchestra.run.smithi150.stdout:hardware devices. 2024-04-15T15:11:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:11:43.399 INFO:teuthology.orchestra.run.smithi150.stdout:This subpackage provides tools for support of all platforms. 2024-04-15T15:11:43.400 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-15T15:11:43.400 INFO:teuthology.task.kernel:Updating grub on smithi150 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:11:43.400 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-15T15:11:44.025 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-04-15T15:11:45.366 INFO:teuthology.orchestra.run.smithi150.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-15T15:11:45.390 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-04-15T15:11:45.391 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-04-15T15:11:45.406 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.HDQbFwuEG3 2024-04-15T15:11:45.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HDQbFwuEG3 2024-04-15T15:11:45.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15 15:11:45 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-15T15:11:45.453 INFO:teuthology.orchestra.run.smithi187.stderr:FINISHED --2024-04-15 15:11:45-- 2024-04-15T15:11:45.454 INFO:teuthology.orchestra.run.smithi187.stderr:Total wall clock time: 1m 38s 2024-04-15T15:11:45.454 INFO:teuthology.orchestra.run.smithi187.stderr:Downloaded: 1 files, 341M in 1m 37s (3.50 MB/s) 2024-04-15T15:11:45.457 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.1... 2024-04-15T15:11:45.457 DEBUG:teuthology.orchestra.run.smithi187:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-15T15:11:45.474 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.HDQbFwuEG3 2024-04-15T15:11:45.614 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.HDQbFwuEG3 is 6KB 2024-04-15T15:11:45.663 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.HDQbFwuEG3 2024-04-15T15:11:45.677 DEBUG:teuthology.orchestra.run.smithi150:> sudo /bin/ls /boot/loader/entries || true 2024-04-15T15:11:45.741 INFO:teuthology.orchestra.run.smithi150.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-15T15:11:45.741 INFO:teuthology.orchestra.run.smithi150.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-15T15:11:45.741 INFO:teuthology.orchestra.run.smithi150.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-15T15:11:45.741 INFO:teuthology.orchestra.run.smithi150.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-15T15:11:45.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:11:45.885 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-04-15T15:11:47.520 INFO:teuthology.orchestra.run.smithi187.stdout:Verifying... ######################################## 2024-04-15T15:11:47.583 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing... ######################################## 2024-04-15T15:11:47.664 INFO:teuthology.orchestra.run.smithi187.stdout:Updating / installing... 2024-04-15T15:11:55.693 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-15T15:12:10.683 INFO:teuthology.orchestra.run.smithi187.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-15T15:12:15.887 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-15T15:12:15.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-15T15:12:15.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:12:33.476 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-15T15:12:33.476 DEBUG:teuthology.orchestra.run.smithi187:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-15T15:12:33.554 INFO:teuthology.orchestra.run.smithi187.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:12:33.554 INFO:teuthology.orchestra.run.smithi187.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:12:33.554 INFO:teuthology.orchestra.run.smithi187.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:12:33.554 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-15T15:12:33.555 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-15T15:12:33.556 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-15T15:12:33.557 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-15T15:12:33.558 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-15T15:12:33.559 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-15T15:12:33.560 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-15T15:12:33.561 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-15T15:12:33.562 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-15T15:12:33.563 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-15T15:12:33.564 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-15T15:12:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-15T15:12:33.566 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-15T15:12:33.567 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-15T15:12:33.568 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-15T15:12:33.569 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-15T15:12:33.570 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-15T15:12:33.571 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-15T15:12:33.572 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-15T15:12:33.573 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-15T15:12:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-15T15:12:33.575 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-15T15:12:33.576 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-15T15:12:33.577 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-15T15:12:33.578 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-15T15:12:33.579 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-15T15:12:33.580 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-15T15:12:33.581 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-15T15:12:33.582 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-15T15:12:33.583 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-15T15:12:33.584 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-15T15:12:33.585 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-15T15:12:33.586 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-15T15:12:33.587 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-15T15:12:33.588 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-15T15:12:33.589 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-15T15:12:33.590 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-15T15:12:33.591 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-15T15:12:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-15T15:12:33.593 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-15T15:12:33.594 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-15T15:12:33.595 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-15T15:12:33.596 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-15T15:12:33.597 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-15T15:12:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-15T15:12:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-15T15:12:33.600 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-15T15:12:33.601 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-15T15:12:33.602 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-15T15:12:33.603 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-15T15:12:33.604 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-15T15:12:33.605 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-15T15:12:33.606 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-15T15:12:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-15T15:12:33.608 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-15T15:12:33.609 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-15T15:12:33.610 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-15T15:12:33.611 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-15T15:12:33.612 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-15T15:12:33.613 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-15T15:12:33.614 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-15T15:12:33.615 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-15T15:12:33.616 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-15T15:12:33.617 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-15T15:12:33.618 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-15T15:12:33.619 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-15T15:12:33.620 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-15T15:12:33.621 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-15T15:12:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-15T15:12:33.623 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-15T15:12:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-15T15:12:33.625 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-15T15:12:33.626 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-15T15:12:33.627 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-15T15:12:33.628 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-15T15:12:33.629 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-15T15:12:33.630 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-15T15:12:33.631 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-15T15:12:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-15T15:12:33.633 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-15T15:12:33.634 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-15T15:12:33.635 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-15T15:12:33.636 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-15T15:12:33.637 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-15T15:12:33.638 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-15T15:12:33.639 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-15T15:12:33.640 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-15T15:12:33.641 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-15T15:12:33.642 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-15T15:12:33.643 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-15T15:12:33.644 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-15T15:12:33.645 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-15T15:12:33.646 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-15T15:12:33.647 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-15T15:12:33.648 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-15T15:12:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-15T15:12:33.650 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-15T15:12:33.651 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-15T15:12:33.652 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-15T15:12:33.653 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-15T15:12:33.654 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-15T15:12:33.655 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-15T15:12:33.656 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-15T15:12:33.657 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-15T15:12:33.658 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-15T15:12:33.659 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-15T15:12:33.660 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-15T15:12:33.661 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-15T15:12:33.662 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-15T15:12:33.663 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-15T15:12:33.664 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-15T15:12:33.665 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-15T15:12:33.666 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-15T15:12:33.667 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-15T15:12:33.668 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-15T15:12:33.669 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-15T15:12:33.670 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-15T15:12:33.671 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-15T15:12:33.672 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-15T15:12:33.673 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-15T15:12:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-15T15:12:33.675 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-15T15:12:33.676 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-15T15:12:33.677 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-15T15:12:33.678 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-15T15:12:33.679 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-15T15:12:33.680 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-15T15:12:33.681 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-15T15:12:33.682 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-15T15:12:33.683 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-15T15:12:33.684 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-15T15:12:33.685 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-15T15:12:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-15T15:12:33.687 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-15T15:12:33.688 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-15T15:12:33.689 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-15T15:12:33.690 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-15T15:12:33.691 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-15T15:12:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-15T15:12:33.693 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-15T15:12:33.694 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-15T15:12:33.695 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-15T15:12:33.696 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-15T15:12:33.697 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-15T15:12:33.698 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-15T15:12:33.698 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-15T15:12:33.698 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-15T15:12:33.698 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-15T15:12:33.701 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:12:33.702 DEBUG:teuthology.orchestra.run.smithi187:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-15T15:12:33.724 INFO:teuthology.orchestra.run.smithi187.stdout:postinstall scriptlet (using /bin/sh): 2024-04-15T15:12:33.724 INFO:teuthology.orchestra.run.smithi187.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-15T15:12:33.724 INFO:teuthology.orchestra.run.smithi187.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-15T15:12:33.724 INFO:teuthology.orchestra.run.smithi187.stdout:fi 2024-04-15T15:12:33.724 INFO:teuthology.orchestra.run.smithi187.stdout:for file in vmlinuz System.map config; do 2024-04-15T15:12:33.724 INFO:teuthology.orchestra.run.smithi187.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-15T15:12:33.724 INFO:teuthology.orchestra.run.smithi187.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout: fi 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:done 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:fi 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:fi 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:12:33.725 INFO:teuthology.orchestra.run.smithi187.stdout:fi 2024-04-15T15:12:33.730 DEBUG:teuthology.orchestra.run.smithi187:> sudo rpm -qi grub2-tools 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Name : grub2-tools 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Epoch : 1 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Version : 2.06 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Release : 68.el9 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: x86_64 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Group : Unspecified 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Size : 8264254 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:License : GPLv3+ 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-15T15:12:33.801 INFO:teuthology.orchestra.run.smithi187.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:Packager : builder@centos.org 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:Vendor : CentOS 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:Summary : Support tools for GRUB. 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:Description : 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-15T15:12:33.802 INFO:teuthology.orchestra.run.smithi187.stdout:hardware devices. 2024-04-15T15:12:33.803 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:12:33.803 INFO:teuthology.orchestra.run.smithi187.stdout:This subpackage provides tools for support of all platforms. 2024-04-15T15:12:33.803 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-15T15:12:33.804 INFO:teuthology.task.kernel:Updating grub on smithi187 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:12:33.804 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-15T15:12:34.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-04-15T15:12:34.354 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2024-04-15T15:12:35.701 INFO:teuthology.orchestra.run.smithi187.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-15T15:12:35.767 INFO:teuthology.orchestra.run.smithi187.stderr:done 2024-04-15T15:12:35.778 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-15T15:12:35.793 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.fbr41MvmmO 2024-04-15T15:12:35.793 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fbr41MvmmO 2024-04-15T15:12:35.866 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.fbr41MvmmO 2024-04-15T15:12:36.007 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.fbr41MvmmO is 6KB 2024-04-15T15:12:36.057 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.fbr41MvmmO 2024-04-15T15:12:36.072 DEBUG:teuthology.orchestra.run.smithi187:> sudo /bin/ls /boot/loader/entries || true 2024-04-15T15:12:36.136 INFO:teuthology.orchestra.run.smithi187.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-15T15:12:36.137 INFO:teuthology.orchestra.run.smithi187.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-15T15:12:36.137 INFO:teuthology.orchestra.run.smithi187.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-15T15:12:36.137 INFO:teuthology.orchestra.run.smithi187.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-15T15:12:36.138 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:12:36.298 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2024-04-15T15:12:36.398 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-04-15T15:12:43.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-15T15:12:43.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:12:46.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-04-15T15:12:58.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-15T15:12:58.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:13:06.306 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-15T15:13:06.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-15T15:13:06.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:13:24.767 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-15T15:13:33.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-15T15:13:33.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:13:36.834 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-15T15:13:48.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-15T15:13:48.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:13:58.422 DEBUG:teuthology.orchestra.remote:timed out 2024-04-15T15:14:07.266 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-15T15:14:13.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-15T15:14:13.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:14:13.847 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-15T15:14:14.540 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-15T15:14:14.541 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-15T15:14:14.541 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-04-15T15:14:14.591 INFO:teuthology.orchestra.run.smithi150.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:14:14.591 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:14:14.591 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-15T15:14:14.591 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-15T15:14:15.592 DEBUG:teuthology.parallel:result is None 2024-04-15T15:14:22.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-15T15:14:22.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:14:22.751 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-15T15:14:23.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-15T15:14:23.422 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-15T15:14:23.422 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-04-15T15:14:23.466 INFO:teuthology.orchestra.run.smithi187.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:14:23.467 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-15T15:14:23.467 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-15T15:14:23.467 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-15T15:14:24.468 DEBUG:teuthology.parallel:result is None 2024-04-15T15:14:24.468 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-15T15:14:24.480 INFO:teuthology.task.internal:Creating test directory... 2024-04-15T15:14:24.480 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-15T15:14:24.488 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-15T15:14:24.536 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-15T15:14:24.544 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-15T15:14:24.550 INFO:teuthology.task.internal:Creating archive directory... 2024-04-15T15:14:24.550 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-15T15:14:24.555 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-15T15:14:24.651 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-15T15:14:24.658 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-15T15:14:24.658 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-15T15:14:24.666 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-15T15:14:24.785 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:14:24.791 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:14:24.854 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:14:24.893 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-15T15:14:24.899 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-15T15:14:24.910 INFO:teuthology.task.internal:Configuring sudo... 2024-04-15T15:14:24.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-15T15:14:24.919 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-15T15:14:25.006 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-15T15:14:25.015 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-15T15:14:25.016 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-15T15:14:25.023 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-15T15:14:25.070 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:14:25.152 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:14:25.255 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:14:25.339 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:14:25.435 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:14:25.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-15T15:14:25.532 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:14:25.606 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-15T15:14:25.699 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:14:25.772 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-15T15:14:25.859 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:14:25.859 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-15T15:14:25.945 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-04-15T15:14:25.953 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-04-15T15:14:26.077 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:14:26.081 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:14:26.466 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-15T15:14:26.474 INFO:teuthology.task.internal:Starting timer... 2024-04-15T15:14:26.474 INFO:teuthology.run_tasks:Running task pcp... 2024-04-15T15:14:26.485 INFO:teuthology.run_tasks:Running task selinux... 2024-04-15T15:14:26.494 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-15T15:14:26.494 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-04-15T15:14:26.614 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-04-15T15:14:26.619 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-04-15T15:14:26.742 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-04-15T15:14:26.746 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-15T15:14:26.746 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-04-15T15:14:26.801 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-04-15T15:14:26.801 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-04-15T15:14:26.864 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-04-15T15:14:26.864 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2024-04-15T15:14:26.864 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:14:26.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:14:26.948 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:14:27.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:14:27.023 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-15T15:14:27.023 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-04-15T15:14:27.105 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-04-15T15:14:27.184 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-15T15:14:27.194 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-15T15:14:27.211 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-15T15:14:27.212 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi150.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-15T15:20:30.975 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-04-15T15:20:30.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-15T15:20:30.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:20:31.123 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-15T15:20:31.282 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-15T15:20:31.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-15T15:20:31.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-15T15:20:31.430 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-15T15:20:31.569 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-15T15:20:31.570 INFO:teuthology.run_tasks:Running task clock... 2024-04-15T15:20:31.584 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-15T15:20:31.584 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-15T15:20:31.585 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:20:31.593 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-15T15:20:31.593 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:20:31.689 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-15T15:20:31.690 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-15T15:20:31.740 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-15T15:20:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-15T15:20:31.834 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-04-15T15:20:31.835 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-04-15T15:20:31.876 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-04-15T15:20:31.877 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-04-15T15:20:31.927 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-15T15:20:31.930 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-15T15:20:31.980 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-15T15:20:31.983 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-15T15:20:32.113 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:20:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:20:32.121 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:20:32.121 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-04-15T15:20:32.121 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:20:32.121 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:20:32.121 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:20:32.121 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:20:32.122 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:20:32.122 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-15T15:20:32.123 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:20:32.123 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:20:32.123 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:20:32.123 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:20:32.124 INFO:teuthology.run_tasks:Running task install... 2024-04-15T15:20:32.132 DEBUG:teuthology.task.install:project ceph 2024-04-15T15:20:32.132 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}} 2024-04-15T15:20:32.133 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'} 2024-04-15T15:20:32.133 INFO:teuthology.task.install:Using flavor: default 2024-04-15T15:20:32.138 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-15T15:20:32.138 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-15T15:20:32.139 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'tag': None, 'wait_for_package': False} 2024-04-15T15:20:32.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:20:32.141 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'tag': None, 'wait_for_package': False} 2024-04-15T15:20:32.142 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:20:32.295 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/ 2024-04-15T15:20:32.295 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2861.g9bf1c4ba 2024-04-15T15:20:32.301 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/ 2024-04-15T15:20:32.301 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2861.g9bf1c4ba 2024-04-15T15:20:32.599 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-15T15:20:32.599 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:20:32.600 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-15T15:20:32.674 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-xlock/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-15T15:20:32.674 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:20:32.675 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-15T15:20:32.699 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-15T15:20:32.699 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-15T15:20:32.779 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-15T15:20:32.779 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-15T15:20:32.833 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-15T15:20:32.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-15T15:20:32.955 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-15T15:20:33.035 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-04-15T15:20:33.039 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-04-15T15:20:33.054 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-15T15:20:33.139 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-04-15T15:20:33.147 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-04-15T15:20:33.393 INFO:teuthology.orchestra.run.smithi187.stdout:74 files removed 2024-04-15T15:20:33.433 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-15T15:20:33.495 INFO:teuthology.orchestra.run.smithi150.stdout:74 files removed 2024-04-15T15:20:33.534 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-15T15:20:34.654 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 117 kB/s | 83 kB 00:00 2024-04-15T15:20:34.856 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 102 kB/s | 83 kB 00:00 2024-04-15T15:20:35.429 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-04-15T15:20:35.599 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-04-15T15:20:36.189 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-04-15T15:20:36.417 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-04-15T15:20:37.033 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - AppStream 8.7 MB/s | 6.7 MB 00:00 2024-04-15T15:20:37.211 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - AppStream 9.3 MB/s | 6.7 MB 00:00 2024-04-15T15:20:38.676 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-04-15T15:20:39.053 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-04-15T15:20:39.822 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-15T15:20:40.314 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-04-15T15:20:42.495 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-15T15:20:42.738 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-15T15:20:47.149 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-15T15:20:47.385 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-15T15:20:49.027 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-15T15:20:49.116 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-15T15:20:50.968 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-15T15:20:51.499 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-04-15T15:20:56.984 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-15T15:20:57.420 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-15T15:20:59.937 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:20:59.938 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:20:59.958 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-15T15:20:59.959 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-15T15:20:59.961 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-15T15:20:59.962 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-15T15:20:59.965 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-15T15:20:59.968 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-15T15:20:59.969 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-15T15:21:00.057 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-04-15T15:21:00.074 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-04-15T15:21:00.074 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-04-15T15:21:00.074 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 6.6 k 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.4 M 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 890 k 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 145 k 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 1.5 M 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 142 k 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 3.5 M 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 7.4 M 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 49 k 2024-04-15T15:21:00.075 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 10 M 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 49 M 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 264 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 762 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 224 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 27 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 18 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 32 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 747 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 127 k 2024-04-15T15:21:00.076 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 162 k 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 320 k 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 297 k 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 99 k 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 85 k 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:21:00.077 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 171 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-15T15:21:00.078 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.4 M 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-15T15:21:00.079 INFO:teuthology.orchestra.run.smithi187.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 21 M 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 34 k 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 2.4 M 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 249 k 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 4.7 M 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 17 M 2024-04-15T15:21:00.080 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 16 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 25 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 164 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-15T15:21:00.081 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 501 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.2 M 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-15T15:21:00.082 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-15T15:21:00.083 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 45 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 135 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-15T15:21:00.084 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-15T15:21:00.085 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-15T15:21:00.086 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-15T15:21:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-15T15:21:00.088 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-15T15:21:00.089 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-04-15T15:21:00.089 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-15T15:21:00.089 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:21:00.089 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-04-15T15:21:00.089 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-04-15T15:21:00.089 INFO:teuthology.orchestra.run.smithi187.stdout:Install 150 Packages 2024-04-15T15:21:00.089 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 24 Packages 2024-04-15T15:21:00.089 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:21:00.090 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 228 M 2024-04-15T15:21:00.090 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-04-15T15:21:00.370 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:21:00.371 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-15T15:21:00.393 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-15T15:21:00.394 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-15T15:21:00.396 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-15T15:21:00.397 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-15T15:21:00.400 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-15T15:21:00.403 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-15T15:21:00.405 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-15T15:21:00.492 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-15T15:21:00.510 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================ 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================ 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 6.6 k 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.4 M 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 890 k 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 145 k 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 1.5 M 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 142 k 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 3.5 M 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 7.4 M 2024-04-15T15:21:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 49 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 10 M 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 49 M 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 264 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 762 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 224 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 27 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 18 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 32 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 747 k 2024-04-15T15:21:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 127 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 162 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 320 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 297 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 99 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 85 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 171 k 2024-04-15T15:21:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-15T15:21:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.4 M 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 3.1 M 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-15T15:21:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 21 M 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 34 k 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 2.4 M 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 249 k 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 4.7 M 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 17 M 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2861.g9bf1c4ba.el8 ceph-noarch 16 k 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 25 k 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-15T15:21:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 164 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-15T15:21:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 501 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 5.2 M 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-15T15:21:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 45 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-2861.g9bf1c4ba.el8 ceph 135 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-15T15:21:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-15T15:21:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-15T15:21:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-15T15:21:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-15T15:21:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================ 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout:Install 150 Packages 2024-04-15T15:21:00.524 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 24 Packages 2024-04-15T15:21:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:21:00.525 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 228 M 2024-04-15T15:21:00.525 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-15T15:21:01.277 INFO:teuthology.orchestra.run.smithi187.stdout:(1/174): ceph-19.0.0-2861.g9bf1c4ba.el8.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-15T15:21:01.639 INFO:teuthology.orchestra.run.smithi150.stdout:(1/174): ceph-19.0.0-2861.g9bf1c4ba.el8.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-15T15:21:01.644 INFO:teuthology.orchestra.run.smithi187.stdout:(2/174): ceph-base-19.0.0-2861.g9bf1c4ba.el8.x8 7.8 MB/s | 5.4 MB 00:00 2024-04-15T15:21:01.711 INFO:teuthology.orchestra.run.smithi187.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 2.2 MB/s | 145 kB 00:00 2024-04-15T15:21:02.070 INFO:teuthology.orchestra.run.smithi187.stdout:(4/174): ceph-mds-19.0.0-2861.g9bf1c4ba.el8.x86 6.6 MB/s | 2.4 MB 00:00 2024-04-15T15:21:02.161 INFO:teuthology.orchestra.run.smithi187.stdout:(5/174): ceph-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 1.0 MB/s | 890 kB 00:00 2024-04-15T15:21:02.312 INFO:teuthology.orchestra.run.smithi187.stdout:(6/174): ceph-mgr-19.0.0-2861.g9bf1c4ba.el8.x86 6.2 MB/s | 1.5 MB 00:00 2024-04-15T15:21:02.505 INFO:teuthology.orchestra.run.smithi150.stdout:(2/174): ceph-base-19.0.0-2861.g9bf1c4ba.el8.x8 4.4 MB/s | 5.4 MB 00:01 2024-04-15T15:21:02.573 INFO:teuthology.orchestra.run.smithi150.stdout:(3/174): ceph-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 954 kB/s | 890 kB 00:00 2024-04-15T15:21:02.606 INFO:teuthology.orchestra.run.smithi150.stdout:(4/174): ceph-immutable-object-cache-19.0.0-286 1.4 MB/s | 145 kB 00:00 2024-04-15T15:21:03.007 INFO:teuthology.orchestra.run.smithi150.stdout:(5/174): ceph-mgr-19.0.0-2861.g9bf1c4ba.el8.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-15T15:21:03.932 INFO:teuthology.orchestra.run.smithi150.stdout:(6/174): ceph-mon-19.0.0-2861.g9bf1c4ba.el8.x86 5.1 MB/s | 4.7 MB 00:00 2024-04-15T15:21:04.027 INFO:teuthology.orchestra.run.smithi150.stdout:(7/174): ceph-mds-19.0.0-2861.g9bf1c4ba.el8.x86 1.6 MB/s | 2.4 MB 00:01 2024-04-15T15:21:04.320 INFO:teuthology.orchestra.run.smithi187.stdout:(7/174): ceph-osd-19.0.0-2861.g9bf1c4ba.el8.x86 8.4 MB/s | 17 MB 00:02 2024-04-15T15:21:04.821 INFO:teuthology.orchestra.run.smithi187.stdout:(8/174): ceph-common-19.0.0-2861.g9bf1c4ba.el8. 5.4 MB/s | 21 MB 00:03 2024-04-15T15:21:04.907 INFO:teuthology.orchestra.run.smithi150.stdout:(8/174): ceph-common-19.0.0-2861.g9bf1c4ba.el8. 5.7 MB/s | 21 MB 00:03 2024-04-15T15:21:04.921 INFO:teuthology.orchestra.run.smithi187.stdout:(9/174): ceph-selinux-19.0.0-2861.g9bf1c4ba.el8 254 kB/s | 25 kB 00:00 2024-04-15T15:21:04.966 INFO:teuthology.orchestra.run.smithi150.stdout:(9/174): ceph-selinux-19.0.0-2861.g9bf1c4ba.el8 439 kB/s | 25 kB 00:00 2024-04-15T15:21:05.230 INFO:teuthology.orchestra.run.smithi187.stdout:(10/174): ceph-mon-19.0.0-2861.g9bf1c4ba.el8.x8 1.5 MB/s | 4.7 MB 00:03 2024-04-15T15:21:05.571 INFO:teuthology.orchestra.run.smithi187.stdout:(11/174): ceph-radosgw-19.0.0-2861.g9bf1c4ba.el 8.0 MB/s | 10 MB 00:01 2024-04-15T15:21:05.605 INFO:teuthology.orchestra.run.smithi187.stdout:(12/174): cephfs-mirror-19.0.0-2861.g9bf1c4ba.e 601 kB/s | 224 kB 00:00 2024-04-15T15:21:05.647 INFO:teuthology.orchestra.run.smithi187.stdout:(13/174): cephfs-shell-19.0.0-2861.g9bf1c4ba.el 364 kB/s | 27 kB 00:00 2024-04-15T15:21:05.697 INFO:teuthology.orchestra.run.smithi187.stdout:(14/174): libcephfs-devel-19.0.0-2861.g9bf1c4ba 348 kB/s | 32 kB 00:00 2024-04-15T15:21:05.805 INFO:teuthology.orchestra.run.smithi187.stdout:(15/174): libcephfs2-19.0.0-2861.g9bf1c4ba.el8. 4.6 MB/s | 747 kB 00:00 2024-04-15T15:21:05.889 INFO:teuthology.orchestra.run.smithi187.stdout:(16/174): librados-devel-19.0.0-2861.g9bf1c4ba. 1.5 MB/s | 127 kB 00:00 2024-04-15T15:21:05.906 INFO:teuthology.orchestra.run.smithi187.stdout:(17/174): libcephsqlite-19.0.0-2861.g9bf1c4ba.e 787 kB/s | 164 kB 00:00 2024-04-15T15:21:06.040 INFO:teuthology.orchestra.run.smithi187.stdout:(18/174): libradosstriper1-19.0.0-2861.g9bf1c4b 3.3 MB/s | 501 kB 00:00 2024-04-15T15:21:06.124 INFO:teuthology.orchestra.run.smithi187.stdout:(19/174): python3-ceph-argparse-19.0.0-2861.g9b 545 kB/s | 45 kB 00:00 2024-04-15T15:21:06.190 INFO:teuthology.orchestra.run.smithi187.stdout:(20/174): python3-ceph-common-19.0.0-2861.g9bf1 2.0 MB/s | 135 kB 00:00 2024-04-15T15:21:06.249 INFO:teuthology.orchestra.run.smithi187.stdout:(21/174): python3-cephfs-19.0.0-2861.g9bf1c4ba. 2.7 MB/s | 162 kB 00:00 2024-04-15T15:21:06.316 INFO:teuthology.orchestra.run.smithi187.stdout:(22/174): python3-rados-19.0.0-2861.g9bf1c4ba.e 4.8 MB/s | 320 kB 00:00 2024-04-15T15:21:06.366 INFO:teuthology.orchestra.run.smithi150.stdout:(10/174): ceph-osd-19.0.0-2861.g9bf1c4ba.el8.x8 6.9 MB/s | 17 MB 00:02 2024-04-15T15:21:06.375 INFO:teuthology.orchestra.run.smithi187.stdout:(23/174): python3-rbd-19.0.0-2861.g9bf1c4ba.el8 5.0 MB/s | 297 kB 00:00 2024-04-15T15:21:06.458 INFO:teuthology.orchestra.run.smithi187.stdout:(24/174): python3-rgw-19.0.0-2861.g9bf1c4ba.el8 1.2 MB/s | 99 kB 00:00 2024-04-15T15:21:06.517 INFO:teuthology.orchestra.run.smithi187.stdout:(25/174): rbd-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 1.5 MB/s | 85 kB 00:00 2024-04-15T15:21:06.525 INFO:teuthology.orchestra.run.smithi150.stdout:(11/174): ceph-radosgw-19.0.0-2861.g9bf1c4ba.el 4.0 MB/s | 10 MB 00:02 2024-04-15T15:21:06.558 INFO:teuthology.orchestra.run.smithi150.stdout:(12/174): cephfs-mirror-19.0.0-2861.g9bf1c4ba.e 1.1 MB/s | 224 kB 00:00 2024-04-15T15:21:06.608 INFO:teuthology.orchestra.run.smithi150.stdout:(13/174): cephfs-shell-19.0.0-2861.g9bf1c4ba.el 328 kB/s | 27 kB 00:00 2024-04-15T15:21:06.634 INFO:teuthology.orchestra.run.smithi150.stdout:(14/174): libcephfs-devel-19.0.0-2861.g9bf1c4ba 425 kB/s | 32 kB 00:00 2024-04-15T15:21:06.709 INFO:teuthology.orchestra.run.smithi150.stdout:(15/174): libcephsqlite-19.0.0-2861.g9bf1c4ba.e 2.2 MB/s | 164 kB 00:00 2024-04-15T15:21:06.793 INFO:teuthology.orchestra.run.smithi150.stdout:(16/174): librados-devel-19.0.0-2861.g9bf1c4ba. 1.5 MB/s | 127 kB 00:00 2024-04-15T15:21:06.843 INFO:teuthology.orchestra.run.smithi150.stdout:(17/174): libcephfs2-19.0.0-2861.g9bf1c4ba.el8. 3.1 MB/s | 747 kB 00:00 2024-04-15T15:21:06.918 INFO:teuthology.orchestra.run.smithi150.stdout:(18/174): libradosstriper1-19.0.0-2861.g9bf1c4b 3.9 MB/s | 501 kB 00:00 2024-04-15T15:21:06.950 INFO:teuthology.orchestra.run.smithi187.stdout:(26/174): rbd-mirror-19.0.0-2861.g9bf1c4ba.el8. 7.1 MB/s | 3.1 MB 00:00 2024-04-15T15:21:06.977 INFO:teuthology.orchestra.run.smithi150.stdout:(19/174): python3-ceph-argparse-19.0.0-2861.g9b 788 kB/s | 45 kB 00:00 2024-04-15T15:21:07.026 INFO:teuthology.orchestra.run.smithi187.stdout:(27/174): rbd-nbd-19.0.0-2861.g9bf1c4ba.el8.x86 2.3 MB/s | 171 kB 00:00 2024-04-15T15:21:07.052 INFO:teuthology.orchestra.run.smithi150.stdout:(20/174): python3-ceph-common-19.0.0-2861.g9bf1 1.8 MB/s | 135 kB 00:00 2024-04-15T15:21:07.084 INFO:teuthology.orchestra.run.smithi187.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2861.g 596 kB/s | 34 kB 00:00 2024-04-15T15:21:07.127 INFO:teuthology.orchestra.run.smithi150.stdout:(21/174): python3-cephfs-19.0.0-2861.g9bf1c4ba. 2.1 MB/s | 162 kB 00:00 2024-04-15T15:21:07.176 INFO:teuthology.orchestra.run.smithi187.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2861.g9bf1c4b 1.5 MB/s | 142 kB 00:00 2024-04-15T15:21:07.211 INFO:teuthology.orchestra.run.smithi150.stdout:(22/174): python3-rados-19.0.0-2861.g9bf1c4ba.e 3.8 MB/s | 320 kB 00:00 2024-04-15T15:21:07.295 INFO:teuthology.orchestra.run.smithi150.stdout:(23/174): python3-rbd-19.0.0-2861.g9bf1c4ba.el8 3.5 MB/s | 297 kB 00:00 2024-04-15T15:21:07.378 INFO:teuthology.orchestra.run.smithi150.stdout:(24/174): python3-rgw-19.0.0-2861.g9bf1c4ba.el8 1.2 MB/s | 99 kB 00:00 2024-04-15T15:21:07.487 INFO:teuthology.orchestra.run.smithi150.stdout:(25/174): rbd-fuse-19.0.0-2861.g9bf1c4ba.el8.x8 797 kB/s | 85 kB 00:00 2024-04-15T15:21:07.718 INFO:teuthology.orchestra.run.smithi187.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2861.g9bf1c 6.5 MB/s | 3.5 MB 00:00 2024-04-15T15:21:07.871 INFO:teuthology.orchestra.run.smithi150.stdout:(26/174): librgw2-19.0.0-2861.g9bf1c4ba.el8.x86 5.1 MB/s | 5.2 MB 00:01 2024-04-15T15:21:08.029 INFO:teuthology.orchestra.run.smithi150.stdout:(27/174): rbd-mirror-19.0.0-2861.g9bf1c4ba.el8. 5.7 MB/s | 3.1 MB 00:00 2024-04-15T15:21:08.063 INFO:teuthology.orchestra.run.smithi150.stdout:(28/174): rbd-nbd-19.0.0-2861.g9bf1c4ba.el8.x86 895 kB/s | 171 kB 00:00 2024-04-15T15:21:08.097 INFO:teuthology.orchestra.run.smithi150.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2861.g 521 kB/s | 34 kB 00:00 2024-04-15T15:21:08.155 INFO:teuthology.orchestra.run.smithi150.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2861.g9bf1c4b 1.5 MB/s | 142 kB 00:00 2024-04-15T15:21:08.739 INFO:teuthology.orchestra.run.smithi150.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2861.g9bf1c 5.5 MB/s | 3.5 MB 00:00 2024-04-15T15:21:08.810 INFO:teuthology.orchestra.run.smithi187.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-04-15T15:21:08.848 INFO:teuthology.orchestra.run.smithi150.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2861.g9b 2.3 MB/s | 249 kB 00:00 2024-04-15T15:21:08.877 INFO:teuthology.orchestra.run.smithi187.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2861.g9b 3.7 MB/s | 249 kB 00:00 2024-04-15T15:21:08.906 INFO:teuthology.orchestra.run.smithi150.stdout:(33/174): ceph-mgr-rook-19.0.0-2861.g9bf1c4ba.e 859 kB/s | 49 kB 00:00 2024-04-15T15:21:08.944 INFO:teuthology.orchestra.run.smithi187.stdout:(33/174): ceph-mgr-rook-19.0.0-2861.g9bf1c4ba.e 748 kB/s | 49 kB 00:00 2024-04-15T15:21:08.973 INFO:teuthology.orchestra.run.smithi150.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2861.g9 250 kB/s | 16 kB 00:00 2024-04-15T15:21:09.011 INFO:teuthology.orchestra.run.smithi187.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2861.g9 249 kB/s | 16 kB 00:00 2024-04-15T15:21:09.057 INFO:teuthology.orchestra.run.smithi150.stdout:(35/174): ceph-volume-19.0.0-2861.g9bf1c4ba.el8 3.1 MB/s | 264 kB 00:00 2024-04-15T15:21:09.120 INFO:teuthology.orchestra.run.smithi187.stdout:(35/174): librgw2-19.0.0-2861.g9bf1c4ba.el8.x86 1.6 MB/s | 5.2 MB 00:03 2024-04-15T15:21:09.154 INFO:teuthology.orchestra.run.smithi187.stdout:(36/174): ceph-volume-19.0.0-2861.g9bf1c4ba.el8 1.8 MB/s | 264 kB 00:00 2024-04-15T15:21:09.182 INFO:teuthology.orchestra.run.smithi150.stdout:(36/174): cephadm-19.0.0-2861.g9bf1c4ba.el8.noa 6.0 MB/s | 762 kB 00:00 2024-04-15T15:21:09.221 INFO:teuthology.orchestra.run.smithi187.stdout:(37/174): cephfs-top-19.0.0-2861.g9bf1c4ba.el8. 276 kB/s | 18 kB 00:00 2024-04-15T15:21:09.241 INFO:teuthology.orchestra.run.smithi150.stdout:(37/174): cephfs-top-19.0.0-2861.g9bf1c4ba.el8. 318 kB/s | 18 kB 00:00 2024-04-15T15:21:09.413 INFO:teuthology.orchestra.run.smithi187.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-04-15T15:21:09.473 INFO:teuthology.orchestra.run.smithi187.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 920 kB 00:00 2024-04-15T15:21:09.475 INFO:teuthology.orchestra.run.smithi150.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 5.6 MB/s | 7.4 MB 00:01 2024-04-15T15:21:09.505 INFO:teuthology.orchestra.run.smithi187.stdout:(40/174): cephadm-19.0.0-2861.g9bf1c4ba.el8.noa 1.9 MB/s | 762 kB 00:00 2024-04-15T15:21:09.530 INFO:teuthology.orchestra.run.smithi187.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-04-15T15:21:09.533 INFO:teuthology.orchestra.run.smithi150.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-04-15T15:21:09.564 INFO:teuthology.orchestra.run.smithi187.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-04-15T15:21:09.589 INFO:teuthology.orchestra.run.smithi187.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 573 kB/s | 32 kB 00:00 2024-04-15T15:21:09.634 INFO:teuthology.orchestra.run.smithi150.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 920 kB 00:00 2024-04-15T15:21:09.659 INFO:teuthology.orchestra.run.smithi150.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 882 kB/s | 106 kB 00:00 2024-04-15T15:21:09.698 INFO:teuthology.orchestra.run.smithi187.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-04-15T15:21:09.709 INFO:teuthology.orchestra.run.smithi150.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 307 kB 00:00 2024-04-15T15:21:09.723 INFO:teuthology.orchestra.run.smithi187.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 139 kB/s | 17 kB 00:00 2024-04-15T15:21:09.734 INFO:teuthology.orchestra.run.smithi150.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 431 kB/s | 32 kB 00:00 2024-04-15T15:21:09.748 INFO:teuthology.orchestra.run.smithi187.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 205 kB/s | 10 kB 00:00 2024-04-15T15:21:09.774 INFO:teuthology.orchestra.run.smithi187.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2024-04-15T15:21:09.807 INFO:teuthology.orchestra.run.smithi187.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 784 kB/s | 44 kB 00:00 2024-04-15T15:21:09.849 INFO:teuthology.orchestra.run.smithi187.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-04-15T15:21:09.891 INFO:teuthology.orchestra.run.smithi187.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2024-04-15T15:21:09.917 INFO:teuthology.orchestra.run.smithi187.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 758 kB/s | 46 kB 00:00 2024-04-15T15:21:09.960 INFO:teuthology.orchestra.run.smithi150.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 77 kB/s | 17 kB 00:00 2024-04-15T15:21:09.975 INFO:teuthology.orchestra.run.smithi187.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 282 kB 00:00 2024-04-15T15:21:10.009 INFO:teuthology.orchestra.run.smithi187.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-04-15T15:21:10.043 INFO:teuthology.orchestra.run.smithi187.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-04-15T15:21:10.060 INFO:teuthology.orchestra.run.smithi150.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-04-15T15:21:10.068 INFO:teuthology.orchestra.run.smithi187.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 34 kB 00:00 2024-04-15T15:21:10.085 INFO:teuthology.orchestra.run.smithi150.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 10 kB 00:00 2024-04-15T15:21:10.177 INFO:teuthology.orchestra.run.smithi150.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-04-15T15:21:10.203 INFO:teuthology.orchestra.run.smithi150.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 382 kB/s | 44 kB 00:00 2024-04-15T15:21:10.235 INFO:teuthology.orchestra.run.smithi187.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-15T15:21:10.278 INFO:teuthology.orchestra.run.smithi150.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2024-04-15T15:21:10.310 INFO:teuthology.orchestra.run.smithi187.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 6.5 MB/s | 1.5 MB 00:00 2024-04-15T15:21:10.328 INFO:teuthology.orchestra.run.smithi150.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 578 kB 00:00 2024-04-15T15:21:10.352 INFO:teuthology.orchestra.run.smithi187.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 817 kB 00:00 2024-04-15T15:21:10.353 INFO:teuthology.orchestra.run.smithi150.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 669 kB/s | 46 kB 00:00 2024-04-15T15:21:10.377 INFO:teuthology.orchestra.run.smithi187.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 132 kB 00:00 2024-04-15T15:21:10.387 INFO:teuthology.orchestra.run.smithi150.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-04-15T15:21:10.421 INFO:teuthology.orchestra.run.smithi150.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2024-04-15T15:21:10.463 INFO:teuthology.orchestra.run.smithi150.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2024-04-15T15:21:10.488 INFO:teuthology.orchestra.run.smithi150.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 516 kB/s | 34 kB 00:00 2024-04-15T15:21:10.570 INFO:teuthology.orchestra.run.smithi187.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2024-04-15T15:21:10.620 INFO:teuthology.orchestra.run.smithi187.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-04-15T15:21:10.645 INFO:teuthology.orchestra.run.smithi187.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 78 kB 00:00 2024-04-15T15:21:10.854 INFO:teuthology.orchestra.run.smithi187.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-04-15T15:21:10.879 INFO:teuthology.orchestra.run.smithi187.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 108 kB/s | 24 kB 00:00 2024-04-15T15:21:10.921 INFO:teuthology.orchestra.run.smithi187.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 447 kB/s | 18 kB 00:00 2024-04-15T15:21:10.946 INFO:teuthology.orchestra.run.smithi187.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 144 kB 00:00 2024-04-15T15:21:10.972 INFO:teuthology.orchestra.run.smithi187.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 647 kB/s | 32 kB 00:00 2024-04-15T15:21:11.147 INFO:teuthology.orchestra.run.smithi187.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2024-04-15T15:21:11.155 INFO:teuthology.orchestra.run.smithi150.stdout:(56/174): ceph-test-19.0.0-2861.g9bf1c4ba.el8.x 7.8 MB/s | 49 MB 00:06 2024-04-15T15:21:11.197 INFO:teuthology.orchestra.run.smithi187.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-04-15T15:21:11.231 INFO:teuthology.orchestra.run.smithi150.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 2.1 MB/s | 1.5 MB 00:00 2024-04-15T15:21:11.231 INFO:teuthology.orchestra.run.smithi187.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.1 MB/s | 174 kB 00:00 2024-04-15T15:21:11.256 INFO:teuthology.orchestra.run.smithi187.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-04-15T15:21:11.282 INFO:teuthology.orchestra.run.smithi187.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 854 kB/s | 41 kB 00:00 2024-04-15T15:21:11.315 INFO:teuthology.orchestra.run.smithi187.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.1 MB/s | 238 kB 00:00 2024-04-15T15:21:11.339 INFO:teuthology.orchestra.run.smithi150.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 5.4 MB/s | 4.7 MB 00:00 2024-04-15T15:21:11.340 INFO:teuthology.orchestra.run.smithi187.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-04-15T15:21:11.372 INFO:teuthology.orchestra.run.smithi150.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 3.8 MB/s | 817 kB 00:00 2024-04-15T15:21:11.374 INFO:teuthology.orchestra.run.smithi187.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-04-15T15:21:11.400 INFO:teuthology.orchestra.run.smithi187.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 844 kB/s | 47 kB 00:00 2024-04-15T15:21:11.406 INFO:teuthology.orchestra.run.smithi150.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 839 kB/s | 132 kB 00:00 2024-04-15T15:21:11.425 INFO:teuthology.orchestra.run.smithi187.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 872 kB/s | 43 kB 00:00 2024-04-15T15:21:11.540 INFO:teuthology.orchestra.run.smithi150.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-15T15:21:11.707 INFO:teuthology.orchestra.run.smithi150.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 260 kB/s | 78 kB 00:00 2024-04-15T15:21:11.825 INFO:teuthology.orchestra.run.smithi187.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 97 kB/s | 39 kB 00:00 2024-04-15T15:21:11.832 INFO:teuthology.orchestra.run.smithi150.stdout:(63/174): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.7 MB 00:00 2024-04-15T15:21:12.074 INFO:teuthology.orchestra.run.smithi150.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 68 kB/s | 24 kB 00:00 2024-04-15T15:21:12.183 INFO:teuthology.orchestra.run.smithi150.stdout:(65/174): python3-babel-2.9.1-2.el9.noarch.rpm 9.1 MB/s | 5.8 MB 00:00 2024-04-15T15:21:12.200 INFO:teuthology.orchestra.run.smithi150.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 51 kB/s | 18 kB 00:00 2024-04-15T15:21:12.233 INFO:teuthology.orchestra.run.smithi150.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 929 kB/s | 144 kB 00:00 2024-04-15T15:21:12.258 INFO:teuthology.orchestra.run.smithi150.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 561 kB/s | 32 kB 00:00 2024-04-15T15:21:12.301 INFO:teuthology.orchestra.run.smithi187.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-15T15:21:12.317 INFO:teuthology.orchestra.run.smithi187.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 84 kB/s | 41 kB 00:00 2024-04-15T15:21:12.359 INFO:teuthology.orchestra.run.smithi187.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-04-15T15:21:12.376 INFO:teuthology.orchestra.run.smithi187.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 601 kB/s | 35 kB 00:00 2024-04-15T15:21:12.402 INFO:teuthology.orchestra.run.smithi187.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-15T15:21:12.452 INFO:teuthology.orchestra.run.smithi187.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-15T15:21:12.469 INFO:teuthology.orchestra.run.smithi187.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 854 kB/s | 56 kB 00:00 2024-04-15T15:21:12.503 INFO:teuthology.orchestra.run.smithi187.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-15T15:21:12.528 INFO:teuthology.orchestra.run.smithi187.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 763 kB/s | 44 kB 00:00 2024-04-15T15:21:12.550 INFO:teuthology.orchestra.run.smithi150.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 4.8 MB 00:00 2024-04-15T15:21:12.553 INFO:teuthology.orchestra.run.smithi187.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-04-15T15:21:12.584 INFO:teuthology.orchestra.run.smithi150.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 622 kB/s | 217 kB 00:00 2024-04-15T15:21:12.587 INFO:teuthology.orchestra.run.smithi187.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-04-15T15:21:12.609 INFO:teuthology.orchestra.run.smithi150.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 506 kB/s | 174 kB 00:00 2024-04-15T15:21:12.634 INFO:teuthology.orchestra.run.smithi150.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 871 kB/s | 69 kB 00:00 2024-04-15T15:21:12.660 INFO:teuthology.orchestra.run.smithi150.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 41 kB 00:00 2024-04-15T15:21:12.679 INFO:teuthology.orchestra.run.smithi187.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.1 MB 00:00 2024-04-15T15:21:12.693 INFO:teuthology.orchestra.run.smithi150.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-04-15T15:21:12.704 INFO:teuthology.orchestra.run.smithi187.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 915 kB/s | 103 kB 00:00 2024-04-15T15:21:12.718 INFO:teuthology.orchestra.run.smithi150.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-04-15T15:21:12.729 INFO:teuthology.orchestra.run.smithi187.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-04-15T15:21:12.752 INFO:teuthology.orchestra.run.smithi150.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2024-04-15T15:21:12.763 INFO:teuthology.orchestra.run.smithi187.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-04-15T15:21:12.777 INFO:teuthology.orchestra.run.smithi150.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 850 kB/s | 47 kB 00:00 2024-04-15T15:21:12.802 INFO:teuthology.orchestra.run.smithi150.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 530 kB/s | 43 kB 00:00 2024-04-15T15:21:12.897 INFO:teuthology.orchestra.run.smithi187.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-15T15:21:12.939 INFO:teuthology.orchestra.run.smithi187.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-15T15:21:12.973 INFO:teuthology.orchestra.run.smithi187.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-15T15:21:13.006 INFO:teuthology.orchestra.run.smithi187.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-15T15:21:13.086 INFO:teuthology.orchestra.run.smithi150.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 140 kB/s | 39 kB 00:00 2024-04-15T15:21:13.111 INFO:teuthology.orchestra.run.smithi150.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 136 kB/s | 41 kB 00:00 2024-04-15T15:21:13.131 INFO:teuthology.orchestra.run.smithi187.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 136 kB/s | 21 kB 00:00 2024-04-15T15:21:13.165 INFO:teuthology.orchestra.run.smithi187.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 161 kB/s | 25 kB 00:00 2024-04-15T15:21:13.324 INFO:teuthology.orchestra.run.smithi187.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.5 MB/s | 245 kB 00:00 2024-04-15T15:21:13.353 INFO:teuthology.orchestra.run.smithi150.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 826 kB/s | 220 kB 00:00 2024-04-15T15:21:13.379 INFO:teuthology.orchestra.run.smithi150.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 136 kB/s | 35 kB 00:00 2024-04-15T15:21:13.407 INFO:teuthology.orchestra.run.smithi187.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.9 MB/s | 248 kB 00:00 2024-04-15T15:21:13.491 INFO:teuthology.orchestra.run.smithi187.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-04-15T15:21:13.533 INFO:teuthology.orchestra.run.smithi187.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 547 kB/s | 22 kB 00:00 2024-04-15T15:21:13.633 INFO:teuthology.orchestra.run.smithi187.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 8.2 MB/s | 4.1 MB 00:00 2024-04-15T15:21:13.637 INFO:teuthology.orchestra.run.smithi150.stdout:(83/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-15T15:21:13.671 INFO:teuthology.orchestra.run.smithi150.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 146 kB/s | 46 kB 00:00 2024-04-15T15:21:13.696 INFO:teuthology.orchestra.run.smithi150.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 983 kB/s | 300 kB 00:00 2024-04-15T15:21:13.700 INFO:teuthology.orchestra.run.smithi187.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-04-15T15:21:13.721 INFO:teuthology.orchestra.run.smithi150.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 708 kB/s | 56 kB 00:00 2024-04-15T15:21:13.747 INFO:teuthology.orchestra.run.smithi150.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-15T15:21:13.772 INFO:teuthology.orchestra.run.smithi150.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 596 kB/s | 44 kB 00:00 2024-04-15T15:21:13.797 INFO:teuthology.orchestra.run.smithi150.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-15T15:21:13.831 INFO:teuthology.orchestra.run.smithi150.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-04-15T15:21:14.006 INFO:teuthology.orchestra.run.smithi150.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.1 MB 00:00 2024-04-15T15:21:14.032 INFO:teuthology.orchestra.run.smithi150.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 441 kB/s | 103 kB 00:00 2024-04-15T15:21:14.057 INFO:teuthology.orchestra.run.smithi150.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 573 kB/s | 124 kB 00:00 2024-04-15T15:21:14.090 INFO:teuthology.orchestra.run.smithi150.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 187 kB 00:00 2024-04-15T15:21:14.293 INFO:teuthology.orchestra.run.smithi187.stdout:(106/174): ceph-test-19.0.0-2861.g9bf1c4ba.el8. 5.2 MB/s | 49 MB 00:09 2024-04-15T15:21:14.793 INFO:teuthology.orchestra.run.smithi187.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 51 kB/s | 19 kB 00:00 2024-04-15T15:21:14.901 INFO:teuthology.orchestra.run.smithi187.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 433 kB/s | 548 kB 00:01 2024-04-15T15:21:14.935 INFO:teuthology.orchestra.run.smithi187.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 250 kB/s | 309 kB 00:01 2024-04-15T15:21:15.119 INFO:teuthology.orchestra.run.smithi187.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 121 kB/s | 26 kB 00:00 2024-04-15T15:21:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 164 kB/s | 187 kB 00:01 2024-04-15T15:21:15.235 INFO:teuthology.orchestra.run.smithi187.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 163 kB/s | 49 kB 00:00 2024-04-15T15:21:15.333 INFO:teuthology.orchestra.run.smithi150.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 157 kB/s | 195 kB 00:01 2024-04-15T15:21:15.344 INFO:teuthology.orchestra.run.smithi187.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-15T15:21:15.370 INFO:teuthology.orchestra.run.smithi187.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 276 kB/s | 67 kB 00:00 2024-04-15T15:21:15.403 INFO:teuthology.orchestra.run.smithi187.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 917 kB/s | 151 kB 00:00 2024-04-15T15:21:15.487 INFO:teuthology.orchestra.run.smithi187.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.2 MB/s | 838 kB 00:00 2024-04-15T15:21:15.520 INFO:teuthology.orchestra.run.smithi187.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-15T15:21:15.537 INFO:teuthology.orchestra.run.smithi187.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 325 kB/s | 43 kB 00:00 2024-04-15T15:21:15.571 INFO:teuthology.orchestra.run.smithi187.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 411 kB/s | 32 kB 00:00 2024-04-15T15:21:15.613 INFO:teuthology.orchestra.run.smithi187.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 202 kB/s | 15 kB 00:00 2024-04-15T15:21:15.638 INFO:teuthology.orchestra.run.smithi187.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-15T15:21:15.666 INFO:teuthology.orchestra.run.smithi150.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 65 kB/s | 21 kB 00:00 2024-04-15T15:21:15.672 INFO:teuthology.orchestra.run.smithi187.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-15T15:21:15.714 INFO:teuthology.orchestra.run.smithi187.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-15T15:21:15.739 INFO:teuthology.orchestra.run.smithi187.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 363 kB/s | 35 kB 00:00 2024-04-15T15:21:15.765 INFO:teuthology.orchestra.run.smithi187.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.1 MB/s | 203 kB 00:00 2024-04-15T15:21:15.766 INFO:teuthology.orchestra.run.smithi150.stdout:(98/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 263 kB/s | 25 kB 00:00 2024-04-15T15:21:15.882 INFO:teuthology.orchestra.run.smithi187.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-15T15:21:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 760 kB/s | 144 kB 00:00 2024-04-15T15:21:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-15T15:21:15.983 INFO:teuthology.orchestra.run.smithi187.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 179 kB/s | 18 kB 00:00 2024-04-15T15:21:16.066 INFO:teuthology.orchestra.run.smithi187.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 216 kB/s | 23 kB 00:00 2024-04-15T15:21:16.094 INFO:teuthology.orchestra.run.smithi187.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 147 kB/s | 19 kB 00:00 2024-04-15T15:21:16.117 INFO:teuthology.orchestra.run.smithi187.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2024-04-15T15:21:16.200 INFO:teuthology.orchestra.run.smithi150.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 9.6 MB/s | 4.1 MB 00:00 2024-04-15T15:21:16.225 INFO:teuthology.orchestra.run.smithi187.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-04-15T15:21:16.242 INFO:teuthology.orchestra.run.smithi150.stdout:(100/174): libgfortran-11.4.1-3.el9.x86_64.rpm 368 kB/s | 803 kB 00:02 2024-04-15T15:21:16.250 INFO:teuthology.orchestra.run.smithi187.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 294 kB/s | 46 kB 00:00 2024-04-15T15:21:16.267 INFO:teuthology.orchestra.run.smithi187.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 526 kB/s | 79 kB 00:00 2024-04-15T15:21:16.275 INFO:teuthology.orchestra.run.smithi150.stdout:(101/174): python3-requests-2.25.1-8.el9.noarch 116 kB/s | 125 kB 00:01 2024-04-15T15:21:16.318 INFO:teuthology.orchestra.run.smithi187.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 640 kB/s | 58 kB 00:00 2024-04-15T15:21:16.319 INFO:teuthology.orchestra.run.smithi150.stdout:(102/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.1 MB/s | 245 kB 00:00 2024-04-15T15:21:16.368 INFO:teuthology.orchestra.run.smithi187.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-15T15:21:16.393 INFO:teuthology.orchestra.run.smithi187.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 132 kB/s | 16 kB 00:00 2024-04-15T15:21:16.419 INFO:teuthology.orchestra.run.smithi187.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 900 kB/s | 90 kB 00:00 2024-04-15T15:21:16.468 INFO:teuthology.orchestra.run.smithi150.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 491 kB/s | 94 kB 00:00 2024-04-15T15:21:16.485 INFO:teuthology.orchestra.run.smithi187.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 313 kB/s | 28 kB 00:00 2024-04-15T15:21:16.501 INFO:teuthology.orchestra.run.smithi150.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-04-15T15:21:16.511 INFO:teuthology.orchestra.run.smithi187.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 267 kB/s | 31 kB 00:00 2024-04-15T15:21:16.536 INFO:teuthology.orchestra.run.smithi187.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-15T15:21:16.578 INFO:teuthology.orchestra.run.smithi187.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 663 kB/s | 59 kB 00:00 2024-04-15T15:21:16.628 INFO:teuthology.orchestra.run.smithi187.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 393 kB/s | 36 kB 00:00 2024-04-15T15:21:16.653 INFO:teuthology.orchestra.run.smithi187.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 439 kB/s | 51 kB 00:00 2024-04-15T15:21:16.678 INFO:teuthology.orchestra.run.smithi187.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-15T15:21:16.718 INFO:teuthology.orchestra.run.smithi150.stdout:(105/174): python3-jinja2-2.11.3-5.el9.noarch.r 522 kB/s | 248 kB 00:00 2024-04-15T15:21:16.720 INFO:teuthology.orchestra.run.smithi187.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 983 kB/s | 90 kB 00:00 2024-04-15T15:21:16.762 INFO:teuthology.orchestra.run.smithi187.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-15T15:21:16.788 INFO:teuthology.orchestra.run.smithi187.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-15T15:21:16.830 INFO:teuthology.orchestra.run.smithi187.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-15T15:21:17.039 INFO:teuthology.orchestra.run.smithi187.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-04-15T15:21:17.052 INFO:teuthology.orchestra.run.smithi150.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-04-15T15:21:17.180 INFO:teuthology.orchestra.run.smithi187.stdout:(151/174): librbd1-19.0.0-2861.g9bf1c4ba.el8.x8 9.0 MB/s | 3.1 MB 00:00 2024-04-15T15:21:17.297 INFO:teuthology.orchestra.run.smithi187.stdout:(152/174): NetworkManager-1.47.4-1.el9.x86_64.r 8.9 MB/s | 2.3 MB 00:00 2024-04-15T15:21:17.402 INFO:teuthology.orchestra.run.smithi150.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 609 kB/s | 548 kB 00:00 2024-04-15T15:21:17.406 INFO:teuthology.orchestra.run.smithi187.stdout:(153/174): NetworkManager-libnm-1.47.4-1.el9.x8 8.6 MB/s | 1.8 MB 00:00 2024-04-15T15:21:17.419 INFO:teuthology.orchestra.run.smithi150.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 53 kB/s | 19 kB 00:00 2024-04-15T15:21:17.423 INFO:teuthology.orchestra.run.smithi187.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 334 kB/s | 41 kB 00:00 2024-04-15T15:21:17.482 INFO:teuthology.orchestra.run.smithi187.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-04-15T15:21:17.486 INFO:teuthology.orchestra.run.smithi150.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 404 kB/s | 309 kB 00:00 2024-04-15T15:21:17.515 INFO:teuthology.orchestra.run.smithi187.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.5 MB/s | 321 kB 00:00 2024-04-15T15:21:17.522 INFO:teuthology.orchestra.run.smithi150.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-04-15T15:21:17.562 INFO:teuthology.orchestra.run.smithi150.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 653 kB/s | 49 kB 00:00 2024-04-15T15:21:17.565 INFO:teuthology.orchestra.run.smithi187.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 890 kB/s | 71 kB 00:00 2024-04-15T15:21:17.591 INFO:teuthology.orchestra.run.smithi187.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 489 kB/s | 36 kB 00:00 2024-04-15T15:21:17.633 INFO:teuthology.orchestra.run.smithi187.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-04-15T15:21:17.662 INFO:teuthology.orchestra.run.smithi150.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-15T15:21:17.675 INFO:teuthology.orchestra.run.smithi187.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-04-15T15:21:17.687 INFO:teuthology.orchestra.run.smithi150.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 405 kB/s | 67 kB 00:00 2024-04-15T15:21:17.700 INFO:teuthology.orchestra.run.smithi187.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 348 kB/s | 23 kB 00:00 2024-04-15T15:21:17.734 INFO:teuthology.orchestra.run.smithi187.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-15T15:21:17.784 INFO:teuthology.orchestra.run.smithi187.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-04-15T15:21:17.796 INFO:teuthology.orchestra.run.smithi150.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-15T15:21:17.818 INFO:teuthology.orchestra.run.smithi187.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-04-15T15:21:17.843 INFO:teuthology.orchestra.run.smithi187.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-04-15T15:21:17.846 INFO:teuthology.orchestra.run.smithi150.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-04-15T15:21:17.868 INFO:teuthology.orchestra.run.smithi187.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 582 kB/s | 29 kB 00:00 2024-04-15T15:21:17.878 INFO:teuthology.orchestra.run.smithi150.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 535 kB/s | 43 kB 00:00 2024-04-15T15:21:17.912 INFO:teuthology.orchestra.run.smithi150.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 500 kB/s | 32 kB 00:00 2024-04-15T15:21:17.935 INFO:teuthology.orchestra.run.smithi187.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-04-15T15:21:17.954 INFO:teuthology.orchestra.run.smithi150.stdout:(118/174): python3-asyncssh-2.13.2-5.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-04-15T15:21:17.971 INFO:teuthology.orchestra.run.smithi150.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-04-15T15:21:18.005 INFO:teuthology.orchestra.run.smithi150.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-15T15:21:18.027 INFO:teuthology.orchestra.run.smithi187.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-04-15T15:21:18.063 INFO:teuthology.orchestra.run.smithi150.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-15T15:21:18.089 INFO:teuthology.orchestra.run.smithi150.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 428 kB/s | 35 kB 00:00 2024-04-15T15:21:18.111 INFO:teuthology.orchestra.run.smithi187.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 13 MB/s | 2.3 MB 00:00 2024-04-15T15:21:18.122 INFO:teuthology.orchestra.run.smithi150.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-15T15:21:18.144 INFO:teuthology.orchestra.run.smithi187.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-04-15T15:21:18.156 INFO:teuthology.orchestra.run.smithi150.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-04-15T15:21:18.170 INFO:teuthology.orchestra.run.smithi187.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 537 kB/s | 31 kB 00:00 2024-04-15T15:21:18.215 INFO:teuthology.orchestra.run.smithi150.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-15T15:21:18.228 INFO:teuthology.orchestra.run.smithi187.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-04-15T15:21:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-04-15T15:21:18.278 INFO:teuthology.orchestra.run.smithi187.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 692 kB/s | 74 kB 00:00 2024-04-15T15:21:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-04-15T15:21:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 142 kB/s | 18 kB 00:00 2024-04-15T15:21:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-04-15T15:21:18.391 INFO:teuthology.orchestra.run.smithi150.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2024-04-15T15:21:18.433 INFO:teuthology.orchestra.run.smithi150.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 298 kB/s | 20 kB 00:00 2024-04-15T15:21:18.437 INFO:teuthology.orchestra.run.smithi187.stdout:(174/174): librados2-19.0.0-2861.g9bf1c4ba.el8. 2.1 MB/s | 3.4 MB 00:01 2024-04-15T15:21:18.452 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:21:18.452 INFO:teuthology.orchestra.run.smithi187.stdout:Total 12 MB/s | 228 MB 00:18 2024-04-15T15:21:18.475 INFO:teuthology.orchestra.run.smithi150.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 556 kB/s | 46 kB 00:00 2024-04-15T15:21:18.509 INFO:teuthology.orchestra.run.smithi150.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-15T15:21:18.576 INFO:teuthology.orchestra.run.smithi150.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-15T15:21:18.601 INFO:teuthology.orchestra.run.smithi150.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 462 kB/s | 58 kB 00:00 2024-04-15T15:21:18.635 INFO:teuthology.orchestra.run.smithi150.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-15T15:21:18.660 INFO:teuthology.orchestra.run.smithi150.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-04-15T15:21:18.685 INFO:teuthology.orchestra.run.smithi150.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-15T15:21:18.719 INFO:teuthology.orchestra.run.smithi150.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-04-15T15:21:18.744 INFO:teuthology.orchestra.run.smithi150.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 370 kB/s | 31 kB 00:00 2024-04-15T15:21:18.778 INFO:teuthology.orchestra.run.smithi150.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-15T15:21:18.811 INFO:teuthology.orchestra.run.smithi150.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-04-15T15:21:18.837 INFO:teuthology.orchestra.run.smithi150.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-04-15T15:21:18.862 INFO:teuthology.orchestra.run.smithi150.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 610 kB/s | 51 kB 00:00 2024-04-15T15:21:18.895 INFO:teuthology.orchestra.run.smithi150.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-15T15:21:18.921 INFO:teuthology.orchestra.run.smithi150.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-15T15:21:18.963 INFO:teuthology.orchestra.run.smithi150.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-15T15:21:19.046 INFO:teuthology.orchestra.run.smithi150.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 133 kB/s | 20 kB 00:00 2024-04-15T15:21:19.226 INFO:teuthology.orchestra.run.smithi150.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 630 kB/s | 192 kB 00:00 2024-04-15T15:21:19.305 INFO:teuthology.orchestra.run.smithi150.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-15T15:21:19.592 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-04-15T15:21:19.673 INFO:teuthology.orchestra.run.smithi150.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2024-04-15T15:21:19.806 INFO:teuthology.orchestra.run.smithi150.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 14 MB/s | 1.8 MB 00:00 2024-04-15T15:21:19.840 INFO:teuthology.orchestra.run.smithi150.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-15T15:21:19.890 INFO:teuthology.orchestra.run.smithi150.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-04-15T15:21:19.940 INFO:teuthology.orchestra.run.smithi150.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-04-15T15:21:19.971 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-04-15T15:21:19.971 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-04-15T15:21:20.107 INFO:teuthology.orchestra.run.smithi150.stdout:(156/174): librados2-19.0.0-2861.g9bf1c4ba.el8. 3.2 MB/s | 3.4 MB 00:01 2024-04-15T15:21:20.649 INFO:teuthology.orchestra.run.smithi150.stdout:(157/174): librbd1-19.0.0-2861.g9bf1c4ba.el8.x8 2.2 MB/s | 3.1 MB 00:01 2024-04-15T15:21:20.799 INFO:teuthology.orchestra.run.smithi150.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 83 kB/s | 71 kB 00:00 2024-04-15T15:21:20.883 INFO:teuthology.orchestra.run.smithi150.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 47 kB/s | 36 kB 00:00 2024-04-15T15:21:21.625 INFO:teuthology.orchestra.run.smithi150.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 197 kB/s | 192 kB 00:00 2024-04-15T15:21:21.650 INFO:teuthology.orchestra.run.smithi150.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 30 kB/s | 23 kB 00:00 2024-04-15T15:21:21.842 INFO:teuthology.orchestra.run.smithi150.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 245 kB/s | 256 kB 00:01 2024-04-15T15:21:22.306 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-04-15T15:21:22.307 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-04-15T15:21:22.543 INFO:teuthology.orchestra.run.smithi150.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 119 kB/s | 108 kB 00:00 2024-04-15T15:21:22.718 INFO:teuthology.orchestra.run.smithi150.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 145 kB/s | 155 kB 00:01 2024-04-15T15:21:22.918 INFO:teuthology.orchestra.run.smithi150.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 126 kB/s | 136 kB 00:01 2024-04-15T15:21:23.385 INFO:teuthology.orchestra.run.smithi150.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 76 kB/s | 64 kB 00:00 2024-04-15T15:21:23.479 INFO:teuthology.orchestra.run.smithi150.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 38 kB/s | 29 kB 00:00 2024-04-15T15:21:25.687 INFO:teuthology.orchestra.run.smithi150.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 448 kB/s | 1.2 MB 00:02 2024-04-15T15:21:27.121 INFO:teuthology.orchestra.run.smithi150.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 325 kB/s | 465 kB 00:01 2024-04-15T15:21:27.488 INFO:teuthology.orchestra.run.smithi150.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 538 kB/s | 2.2 MB 00:04 2024-04-15T15:21:27.838 INFO:teuthology.orchestra.run.smithi150.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 536 kB/s | 2.3 MB 00:04 2024-04-15T15:21:27.938 INFO:teuthology.orchestra.run.smithi150.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 38 kB/s | 31 kB 00:00 2024-04-15T15:21:28.197 INFO:teuthology.orchestra.run.smithi150.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 258 kB/s | 74 kB 00:00 2024-04-15T15:21:29.672 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-04-15T15:21:29.783 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-15T15:21:29.829 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-15T15:21:29.884 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:21:29.928 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:21:30.015 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-15T15:21:30.143 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-15T15:21:30.200 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-15T15:21:30.232 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-15T15:21:30.272 INFO:teuthology.orchestra.run.smithi150.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 395 kB/s | 1.1 MB 00:02 2024-04-15T15:21:30.282 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-15T15:21:30.282 INFO:teuthology.orchestra.run.smithi150.stdout:Total 7.6 MB/s | 228 MB 00:29 2024-04-15T15:21:30.325 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-15T15:21:30.733 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-15T15:21:30.780 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:21:30.957 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:21:30.990 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:21:31.123 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:21:31.155 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:21:31.230 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-15T15:21:31.393 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-15T15:21:31.501 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-15T15:21:31.562 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-15T15:21:31.587 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:21:31.714 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:21:31.746 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:21:31.771 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-15T15:21:31.771 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-15T15:21:31.834 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:21:31.880 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-15T15:21:31.898 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:21:31.957 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:21:32.312 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:21:32.457 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:21:32.499 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-15T15:21:32.815 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-15T15:21:33.021 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:21:33.111 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:21:33.306 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:21:33.351 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:21:33.705 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:21:33.877 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-15T15:21:34.001 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-15T15:21:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-15T15:21:34.068 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-15T15:21:34.124 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-15T15:21:34.228 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-15T15:21:34.228 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-15T15:21:34.236 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-15T15:21:34.297 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-15T15:21:34.339 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-15T15:21:34.428 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-15T15:21:34.470 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-15T15:21:34.530 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-15T15:21:34.575 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-15T15:21:34.597 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-15T15:21:34.690 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-15T15:21:34.727 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 39/198 2024-04-15T15:21:34.777 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 40/198 2024-04-15T15:21:34.808 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 41/198 2024-04-15T15:21:34.943 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-15T15:21:35.005 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-15T15:21:35.097 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-15T15:21:35.275 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-15T15:21:35.376 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-15T15:21:36.071 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-15T15:21:36.162 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-15T15:21:36.274 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-15T15:21:36.339 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:21:36.365 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:21:36.425 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-15T15:21:36.503 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-15T15:21:36.561 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 53/198 2024-04-15T15:21:36.607 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-15T15:21:37.059 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-15T15:21:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-15T15:21:37.370 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-15T15:21:37.426 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-15T15:21:37.618 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-15T15:21:37.675 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-15T15:21:37.748 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-15T15:21:37.834 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-15T15:21:37.875 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-15T15:21:37.913 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-15T15:21:37.981 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-15T15:21:38.023 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-15T15:21:38.086 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-15T15:21:38.155 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-15T15:21:38.350 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-15T15:21:38.403 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-15T15:21:38.441 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-15T15:21:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-15T15:21:38.628 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-15T15:21:38.673 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-15T15:21:38.718 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-15T15:21:38.756 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-15T15:21:38.789 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-15T15:21:38.947 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-15T15:21:39.121 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-15T15:21:39.260 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-15T15:21:39.323 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-15T15:21:39.385 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-15T15:21:39.461 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-15T15:21:39.718 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-15T15:21:39.778 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-15T15:21:39.948 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-15T15:21:39.998 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-15T15:21:40.040 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-15T15:21:40.079 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:21:40.119 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-15T15:21:40.165 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-15T15:21:40.223 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-15T15:21:40.273 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-15T15:21:40.314 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-15T15:21:40.477 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-15T15:21:40.511 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-15T15:21:40.547 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-15T15:21:40.609 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-15T15:21:40.649 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-15T15:21:40.792 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-15T15:21:40.912 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-15T15:21:40.962 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-15T15:21:41.007 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-15T15:21:41.062 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-15T15:21:41.202 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-15T15:21:41.264 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-15T15:21:41.431 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-15T15:21:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-15T15:21:41.586 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-15T15:21:41.632 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:21:41.677 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-15T15:21:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-15T15:21:41.874 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-15T15:21:41.924 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-15T15:21:41.956 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-15T15:21:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-15T15:21:42.217 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-15T15:21:42.312 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-15T15:21:42.355 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-15T15:21:42.413 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-15T15:21:42.470 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-15T15:21:42.513 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:21:42.527 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-15T15:21:42.565 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-15T15:21:42.656 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:21:42.680 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:21:42.793 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-15T15:21:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:21:42.828 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-15T15:21:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:21:42.888 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-15T15:21:42.922 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-15T15:21:43.149 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-15T15:21:43.201 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-15T15:21:43.219 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:21:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:21:43.412 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:21:43.502 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 17/198 2024-04-15T15:21:43.555 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-15T15:21:43.580 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:21:43.654 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-15T15:21:43.722 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-15T15:21:43.856 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-15T15:21:43.957 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:21:43.991 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-15T15:21:44.031 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-15T15:21:44.344 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-15T15:21:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:21:44.600 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-15T15:21:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:21:44.849 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:21:45.183 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-15T15:21:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-15T15:21:45.386 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-15T15:21:45.474 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-15T15:21:45.505 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-15T15:21:45.556 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-15T15:21:45.556 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-15T15:21:45.556 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-15T15:21:45.568 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-15T15:21:45.568 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-15T15:21:45.616 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-15T15:21:45.722 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-15T15:21:45.789 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-15T15:21:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-15T15:21:45.915 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-15T15:21:45.931 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-15T15:21:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-15T15:21:45.965 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:21:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-15T15:21:46.048 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-15T15:21:46.081 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-15T15:21:46.093 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:21:46.129 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 125/198 2024-04-15T15:21:46.176 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-15T15:21:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-15T15:21:46.227 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 39/198 2024-04-15T15:21:46.279 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 40/198 2024-04-15T15:21:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 41/198 2024-04-15T15:21:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-15T15:21:46.495 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-15T15:21:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-15T15:21:46.698 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-15T15:21:46.725 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:21:46.772 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-15T15:21:46.824 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:21:46.840 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-15T15:21:46.851 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 129/198 2024-04-15T15:21:47.537 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-15T15:21:47.638 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-15T15:21:47.743 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-15T15:21:47.797 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:21:47.823 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-15T15:21:47.870 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-15T15:21:47.935 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-15T15:21:47.987 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 53/198 2024-04-15T15:21:48.037 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-15T15:21:48.438 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-15T15:21:48.619 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-15T15:21:48.717 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-15T15:21:48.781 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-15T15:21:48.983 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-15T15:21:49.032 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-15T15:21:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-15T15:21:49.182 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-15T15:21:49.222 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-15T15:21:49.260 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-15T15:21:49.320 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-15T15:21:49.359 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-15T15:21:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-15T15:21:49.469 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-15T15:21:49.515 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:21:49.565 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:21:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-15T15:21:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-15T15:21:49.746 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-15T15:21:49.790 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-15T15:21:49.923 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-15T15:21:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-15T15:21:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-15T15:21:50.044 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-15T15:21:50.076 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-15T15:21:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-15T15:21:50.207 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:21:50.251 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:21:50.356 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-15T15:21:50.407 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:21:50.408 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-15T15:21:50.408 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-15T15:21:50.408 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:21:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:21:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-15T15:21:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-15T15:21:50.596 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-15T15:21:50.661 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-15T15:21:50.886 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-15T15:21:50.941 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-15T15:21:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-15T15:21:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-15T15:21:51.186 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-15T15:21:51.217 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:21:51.256 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-15T15:21:51.302 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-15T15:21:51.360 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-15T15:21:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-15T15:21:51.496 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-15T15:21:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-15T15:21:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-15T15:21:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-15T15:21:51.765 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-15T15:21:51.806 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-15T15:21:51.948 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-15T15:21:52.068 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-15T15:21:52.118 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-15T15:21:52.154 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-15T15:21:52.207 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-15T15:21:52.347 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-15T15:21:52.412 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-15T15:21:53.372 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-15T15:21:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-15T15:21:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-15T15:21:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-15T15:21:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-15T15:21:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-15T15:21:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-15T15:21:53.933 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-15T15:21:54.026 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-15T15:21:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-15T15:21:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-15T15:21:56.544 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-15T15:21:56.645 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-15T15:21:56.683 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-15T15:21:56.714 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-15T15:21:56.754 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-15T15:21:57.128 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-15T15:21:57.161 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:21:57.298 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 124/198 2024-04-15T15:21:57.325 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 125/198 2024-04-15T15:21:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-15T15:21:57.907 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-15T15:21:57.939 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:21:58.030 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 128/198 2024-04-15T15:21:58.057 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 129/198 2024-04-15T15:22:01.147 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:22:01.187 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:22:01.831 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 130/198 2024-04-15T15:22:01.874 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:22:02.051 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 131/198 2024-04-15T15:22:02.051 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-15T15:22:02.051 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-15T15:22:02.051 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:02.108 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:22:03.382 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:22:03.382 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-04-15T15:22:03.382 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-04-15T15:22:03.382 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-04-15T15:22:03.383 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-04-15T15:22:03.383 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-04-15T15:22:03.383 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-04-15T15:22:03.383 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-04-15T15:22:03.383 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:03.632 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:22:03.686 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:22:03.686 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:03.686 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-15T15:22:03.686 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:05.351 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:22:05.429 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:22:05.430 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:05.430 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-15T15:22:05.430 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:22:05.430 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:22:05.430 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:05.685 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:22:05.754 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:22:05.754 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:05.754 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-15T15:22:05.754 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:22:05.754 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:22:05.754 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:06.244 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:22:06.311 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:22:06.311 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:06.311 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-15T15:22:06.312 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:22:06.312 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:22:06.312 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:06.438 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-15T15:22:07.057 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:22:07.088 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:22:07.138 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 139/198 2024-04-15T15:22:07.237 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 140/198 2024-04-15T15:22:07.267 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:22:08.570 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:22:08.611 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:22:09.710 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:22:09.736 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:22:09.880 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:22:10.052 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 144/198 2024-04-15T15:22:10.079 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:22:10.151 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:22:10.151 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:10.152 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-15T15:22:10.152 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:22:10.152 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:22:10.152 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:10.205 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:22:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:22:10.296 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 147/198 2024-04-15T15:22:11.267 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-15T15:22:11.299 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:22:11.374 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:22:11.374 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:11.374 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-15T15:22:11.374 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:22:11.374 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:22:11.374 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:11.424 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:22:11.491 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:22:11.491 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:11.492 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-15T15:22:11.492 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:11.567 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:22:11.642 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:22:11.642 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:11.642 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-15T15:22:11.642 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:22:11.643 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:22:11.643 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:12.026 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:22:12.098 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:22:12.099 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:12.099 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-15T15:22:12.099 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:22:12.099 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:22:12.099 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:14.963 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 132/198 2024-04-15T15:22:14.963 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-04-15T15:22:14.963 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-04-15T15:22:14.963 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-04-15T15:22:14.963 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-04-15T15:22:14.964 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-04-15T15:22:14.964 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-04-15T15:22:14.964 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-04-15T15:22:14.964 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:22:15.230 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 133/198 2024-04-15T15:22:15.230 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:15.230 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-15T15:22:15.231 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:16.850 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:22:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 134/198 2024-04-15T15:22:16.928 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:16.928 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-15T15:22:16.928 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:22:16.928 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-15T15:22:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:17.059 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:22:17.114 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 154/198 2024-04-15T15:22:17.172 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 155/198 2024-04-15T15:22:17.175 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:22:17.212 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-15T15:22:17.243 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 135/198 2024-04-15T15:22:17.244 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:17.244 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-15T15:22:17.244 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:22:17.244 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-15T15:22:17.244 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:17.504 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 157/198 2024-04-15T15:22:17.633 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-15T15:22:17.680 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 159/198 2024-04-15T15:22:17.726 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:22:17.732 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-15T15:22:17.794 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 136/198 2024-04-15T15:22:17.794 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:17.794 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-15T15:22:17.794 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:22:17.795 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-15T15:22:17.795 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:17.805 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-15T15:22:17.858 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-15T15:22:17.902 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 163/198 2024-04-15T15:22:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-15T15:22:17.946 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-15T15:22:17.999 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-15T15:22:18.096 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-15T15:22:18.231 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-15T15:22:18.272 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-15T15:22:18.542 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:22:18.579 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 138/198 2024-04-15T15:22:18.629 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 139/198 2024-04-15T15:22:18.729 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 140/198 2024-04-15T15:22:18.758 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:22:19.130 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:22:19.178 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:22:19.235 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:22:20.060 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 141/198 2024-04-15T15:22:20.110 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:22:20.569 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 170/198 2024-04-15T15:22:20.674 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-15T15:22:20.718 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-15T15:22:20.834 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-15T15:22:20.904 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-15T15:22:20.937 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-15T15:22:20.971 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-15T15:22:21.014 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-15T15:22:21.014 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:22:21.167 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:22:21.216 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 142/198 2024-04-15T15:22:21.217 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-15T15:22:21.217 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:22:21.251 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:22:21.355 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:22:21.398 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 143/198 2024-04-15T15:22:21.405 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-15T15:22:21.455 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-15T15:22:21.487 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-15T15:22:21.519 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-15T15:22:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 144/198 2024-04-15T15:22:21.588 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-15T15:22:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:22:21.667 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-15T15:22:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 145/198 2024-04-15T15:22:21.680 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:21.680 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-15T15:22:21.680 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:22:21.680 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-15T15:22:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:21.716 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-15T15:22:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:22:21.749 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-15T15:22:21.775 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 146/198 2024-04-15T15:22:21.782 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-15T15:22:21.835 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 147/198 2024-04-15T15:22:21.841 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:22:21.875 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-15T15:22:21.912 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:22:21.912 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:22:22.233 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:22:22.234 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:22:22.373 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:22:22.423 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:22:22.453 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-15T15:22:22.489 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-15T15:22:22.529 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-15T15:22:22.648 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:22:22.810 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-15T15:22:22.848 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:22:22.882 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:22:22.882 INFO:teuthology.orchestra.run.smithi187.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-15T15:22:22.882 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:22.916 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 149/198 2024-04-15T15:22:22.916 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:22.916 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-15T15:22:22.916 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:22:22.916 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-15T15:22:22.916 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:22.965 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:22:23.034 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 150/198 2024-04-15T15:22:23.034 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:23.034 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-15T15:22:23.034 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:22:23.178 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:22:23.178 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:23.179 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-15T15:22:23.179 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:22:23.179 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-15T15:22:23.179 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:23.558 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:22:23.629 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 152/198 2024-04-15T15:22:23.629 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-15T15:22:23.629 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-15T15:22:23.630 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:22:23.630 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-15T15:22:23.630 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:28.575 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:22:28.632 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 154/198 2024-04-15T15:22:28.676 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 155/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 1/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 3/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 4/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 5/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 6/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 7/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 8/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 9/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:22:28.683 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 13/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 14/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 15/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 17/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 18/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 19/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 20/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 21/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 22/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 23/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 24/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 25/198 2024-04-15T15:22:28.684 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 26/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 27/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 28/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 29/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 30/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 31/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 32/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 33/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 34/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 35/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 36/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 37/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 38/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 39/198 2024-04-15T15:22:28.685 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-15T15:22:28.686 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-15T15:22:28.687 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-15T15:22:28.688 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:22:28.689 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-15T15:22:28.690 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-15T15:22:28.691 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-15T15:22:28.692 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-15T15:22:28.693 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-15T15:22:28.694 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-15T15:22:28.695 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-15T15:22:28.696 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:22:28.697 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-15T15:22:28.698 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-15T15:22:28.698 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-15T15:22:28.698 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:22:28.698 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-15T15:22:28.698 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-15T15:22:28.698 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-15T15:22:28.711 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-15T15:22:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 157/198 2024-04-15T15:22:29.116 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-15T15:22:29.181 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 159/198 2024-04-15T15:22:29.233 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-15T15:22:29.305 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-15T15:22:29.350 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-15T15:22:29.381 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 163/198 2024-04-15T15:22:29.421 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-15T15:22:29.474 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-15T15:22:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-15T15:22:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-15T15:22:29.697 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-15T15:22:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:22:30.627 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:22:30.693 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-15T15:22:31.218 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-15T15:22:31.218 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:31.218 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-04-15T15:22:31.219 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-15T15:22:31.219 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-15T15:22:31.219 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-15T15:22:31.219 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-15T15:22:31.219 INFO:teuthology.orchestra.run.smithi187.stdout: acl-2.3.1-4.el9.x86_64 2024-04-15T15:22:31.219 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-15T15:22:31.220 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-15T15:22:31.220 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-15T15:22:31.220 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-15T15:22:31.220 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-15T15:22:31.220 INFO:teuthology.orchestra.run.smithi187.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.221 INFO:teuthology.orchestra.run.smithi187.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.221 INFO:teuthology.orchestra.run.smithi187.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.221 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.221 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.221 INFO:teuthology.orchestra.run.smithi187.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.221 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.221 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-15T15:22:31.222 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-15T15:22:31.222 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.222 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.222 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.222 INFO:teuthology.orchestra.run.smithi187.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-15T15:22:31.222 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-15T15:22:31.223 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-04-15T15:22:31.223 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-15T15:22:31.223 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.2-8.el9.noarch 2024-04-15T15:22:31.223 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.7.4-5.el9.x86_64 2024-04-15T15:22:31.223 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-15T15:22:31.223 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.224 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.224 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.224 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.224 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.224 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.224 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.225 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.225 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.225 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.225 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.225 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.225 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.225 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.226 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.226 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.226 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.226 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.226 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.226 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.227 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.227 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.227 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.227 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:31.227 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-15T15:22:31.227 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.4-9.el9.x86_64 2024-04-15T15:22:31.227 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-15T15:22:31.228 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-15T15:22:31.228 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-15T15:22:31.228 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-15T15:22:31.228 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-15T15:22:31.228 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-15T15:22:31.228 INFO:teuthology.orchestra.run.smithi187.stdout: jq-1.6-16.el9.x86_64 2024-04-15T15:22:31.229 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-15T15:22:31.229 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-15T15:22:31.229 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-15T15:22:31.229 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.229 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.230 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.230 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-15T15:22:31.230 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-15T15:22:31.230 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-15T15:22:31.230 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-15T15:22:31.230 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-15T15:22:31.231 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.231 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.231 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-15T15:22:31.231 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.231 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-15T15:22:31.231 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-15T15:22:31.232 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-15T15:22:31.232 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-15T15:22:31.232 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-15T15:22:31.232 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-04-15T15:22:31.232 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-15T15:22:31.232 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-15T15:22:31.233 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-15T15:22:31.233 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-15T15:22:31.233 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-15T15:22:31.233 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-15T15:22:31.233 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-15T15:22:31.233 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-13.14-1.el9.x86_64 2024-04-15T15:22:31.234 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-15T15:22:31.234 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-15T15:22:31.234 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-15T15:22:31.234 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-15T15:22:31.234 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-15T15:22:31.234 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-15T15:22:31.235 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-15T15:22:31.235 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-15T15:22:31.235 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-15T15:22:31.235 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-15T15:22:31.235 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.235 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.236 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.236 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-15T15:22:31.236 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-15T15:22:31.236 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-15T15:22:31.236 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-15T15:22:31.237 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-15T15:22:31.237 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-15T15:22:31.237 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-15T15:22:31.237 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-15T15:22:31.237 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-15T15:22:31.237 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-15T15:22:31.237 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-15T15:22:31.238 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-15T15:22:31.238 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-15T15:22:31.238 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-15T15:22:31.238 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-15T15:22:31.238 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-15T15:22:31.238 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-15T15:22:31.239 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-15T15:22:31.239 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-15T15:22:31.239 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-15T15:22:31.239 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-15T15:22:31.239 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-15T15:22:31.239 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-15T15:22:31.239 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-15T15:22:31.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-15T15:22:31.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-15T15:22:31.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-15T15:22:31.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-15T15:22:31.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-15T15:22:31.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-15T15:22:31.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-04-15T15:22:31.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-15T15:22:31.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-15T15:22:31.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-15T15:22:31.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-15T15:22:31.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-15T15:22:31.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-15T15:22:31.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-15T15:22:31.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-15T15:22:31.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-15T15:22:31.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-15T15:22:31.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-15T15:22:31.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-15T15:22:31.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-15T15:22:31.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-15T15:22:31.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-15T15:22:31.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-15T15:22:31.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-15T15:22:31.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-15T15:22:31.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-15T15:22:31.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-15T15:22:31.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-15T15:22:31.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-15T15:22:31.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-15T15:22:31.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-15T15:22:31.245 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-15T15:22:31.245 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-15T15:22:31.245 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-15T15:22:31.246 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.246 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.246 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:31.246 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-3.el9.x86_64 2024-04-15T15:22:31.246 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-15T15:22:31.246 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-15T15:22:31.247 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-15T15:22:31.247 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-15T15:22:31.247 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-15T15:22:31.248 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-15T15:22:31.248 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:31.248 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-04-15T15:22:32.029 DEBUG:teuthology.parallel:result is None 2024-04-15T15:22:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 170/198 2024-04-15T15:22:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-15T15:22:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-15T15:22:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-15T15:22:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-15T15:22:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-15T15:22:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-15T15:22:32.436 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-15T15:22:32.436 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:22:32.584 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-15T15:22:32.625 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-15T15:22:32.626 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:22:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-15T15:22:33.218 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-15T15:22:33.270 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-15T15:22:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-15T15:22:33.335 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-15T15:22:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-15T15:22:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-15T15:22:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-15T15:22:33.599 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-15T15:22:33.632 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-15T15:22:33.666 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:22:33.699 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-15T15:22:33.736 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:22:33.736 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:22:34.075 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-15T15:22:34.075 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:22:34.214 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-15T15:22:34.265 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:22:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-15T15:22:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-15T15:22:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-15T15:22:34.498 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:22:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-15T15:22:34.763 INFO:teuthology.orchestra.run.smithi150.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-15T15:22:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:39.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 1/198 2024-04-15T15:22:39.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2/198 2024-04-15T15:22:39.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 3/198 2024-04-15T15:22:39.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 4/198 2024-04-15T15:22:39.970 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c 5/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 6/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 7/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 8/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 9/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 10/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 11/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 12/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 13/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 14/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_ 15/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 16/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 17/198 2024-04-15T15:22:39.971 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 18/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86 19/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 20/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el 21/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8. 22/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_6 23/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 24/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 25/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 26/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 27/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 28/198 2024-04-15T15:22:39.972 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 29/198 2024-04-15T15:22:39.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba. 30/198 2024-04-15T15:22:39.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noa 31/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.n 32/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf 33/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el 34/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 35/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.e 36/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 37/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 38/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 39/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-15T15:22:39.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-15T15:22:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-15T15:22:39.991 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-15T15:22:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-15T15:22:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-15T15:22:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-15T15:22:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-15T15:22:39.996 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-15T15:22:39.997 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 151/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-15T15:22:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 153/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-15T15:22:39.999 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-15T15:22:40.000 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-15T15:22:40.001 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-15T15:22:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-15T15:22:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-15T15:22:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-15T15:22:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-15T15:22:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-15T15:22:40.002 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-15T15:22:42.526 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-15T15:22:42.526 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:42.526 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-04-15T15:22:42.526 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-15T15:22:42.527 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-15T15:22:42.527 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-15T15:22:42.527 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-15T15:22:42.527 INFO:teuthology.orchestra.run.smithi150.stdout: acl-2.3.1-4.el9.x86_64 2024-04-15T15:22:42.527 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-15T15:22:42.528 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-15T15:22:42.528 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-15T15:22:42.528 INFO:teuthology.orchestra.run.smithi150.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-15T15:22:42.528 INFO:teuthology.orchestra.run.smithi150.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-15T15:22:42.528 INFO:teuthology.orchestra.run.smithi150.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.528 INFO:teuthology.orchestra.run.smithi150.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.529 INFO:teuthology.orchestra.run.smithi150.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.529 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.529 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.529 INFO:teuthology.orchestra.run.smithi150.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.529 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.529 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-15T15:22:42.530 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-15T15:22:42.530 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.530 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.530 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.530 INFO:teuthology.orchestra.run.smithi150.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-15T15:22:42.531 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-15T15:22:42.531 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-15T15:22:42.531 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-15T15:22:42.531 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-04-15T15:22:42.531 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-04-15T15:22:42.531 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-15T15:22:42.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.533 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.533 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.533 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.533 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.533 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.534 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.534 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.534 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.534 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.534 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.534 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.535 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.535 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.535 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.535 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.535 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.535 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-2861.g9bf1c4ba.el8.noarch 2024-04-15T15:22:42.535 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-15T15:22:42.536 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-04-15T15:22:42.536 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-15T15:22:42.536 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-15T15:22:42.536 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-15T15:22:42.536 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-15T15:22:42.537 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-15T15:22:42.537 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-15T15:22:42.537 INFO:teuthology.orchestra.run.smithi150.stdout: jq-1.6-16.el9.x86_64 2024-04-15T15:22:42.537 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-15T15:22:42.537 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-15T15:22:42.537 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-15T15:22:42.537 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.538 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.538 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.538 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-15T15:22:42.538 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-15T15:22:42.538 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-15T15:22:42.538 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-15T15:22:42.538 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-15T15:22:42.539 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.539 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.539 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-15T15:22:42.539 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.539 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-15T15:22:42.539 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-15T15:22:42.540 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-15T15:22:42.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-15T15:22:42.543 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-15T15:22:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-15T15:22:42.545 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-2861.g9bf1c4ba.el8.x86_64 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-04-15T15:22:42.546 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-15T15:22:42.547 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-15T15:22:42.547 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-15T15:22:42.547 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-15T15:22:42.547 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-15T15:22:42.547 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-15T15:22:42.547 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:42.547 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-15T15:22:43.177 DEBUG:teuthology.parallel:result is None 2024-04-15T15:22:43.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:22:43.337 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-15T15:22:43.393 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:22:43.394 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:22:43.394 INFO:teuthology.task.install:The correct ceph version 19.0.0-2861.g9bf1c4ba is installed. 2024-04-15T15:22:43.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9bf1c4ba967dd24077be6e5b87edda4ca87a6e14 2024-04-15T15:22:43.538 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-15T15:22:43.609 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:22:43.609 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2861.g9bf1c4ba.el8 2024-04-15T15:22:43.610 INFO:teuthology.task.install:The correct ceph version 19.0.0-2861.g9bf1c4ba is installed. 2024-04-15T15:22:43.611 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-15T15:22:43.612 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:43.612 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-15T15:22:43.700 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:22:43.700 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-15T15:22:43.805 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-15T15:22:43.806 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:43.806 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-04-15T15:22:43.887 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-15T15:22:44.000 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:22:44.000 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-04-15T15:22:44.100 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-15T15:22:44.237 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-15T15:22:44.237 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:44.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-15T15:22:44.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-15T15:22:44.428 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:22:44.428 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-15T15:22:44.523 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-15T15:22:44.661 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-15T15:22:44.662 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:44.662 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-04-15T15:22:44.736 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-15T15:22:44.854 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:22:44.854 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-04-15T15:22:44.955 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-15T15:22:45.044 INFO:teuthology.run_tasks:Running task ceph... 2024-04-15T15:22:45.152 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-15T15:22:45.152 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2024-04-15T15:22:45.157 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2024-04-15T15:22:45.251 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-15T15:22:45.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-15T15:22:45.256 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-15T15:22:45.344 INFO:tasks.ceph:Creating extra log directories... 2024-04-15T15:22:45.345 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-15T15:22:45.349 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-15T15:22:45.447 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-15T15:22:45.447 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-15T15:22:45.447 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-15T15:22:45.448 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:45.448 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-15T15:22:45.491 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-15T15:22:45.701 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-15T15:22:45.796 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:22:45.796 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-15T15:22:45.851 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-15T15:22:46.113 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-15T15:22:46.215 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-15T15:22:46.215 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-15T15:22:46.216 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136', 'branch': 'wip-lusov-quiesce-xlock', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce}', 'email': None, 'first_in_suite': False, 'job_id': '7657136', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-quiesce-xlock'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-quiesce-xlock', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5929, 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-lusov-quiesce-xlock', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFuSZ/DP/wyAzuGlxUJi84I4XzTKzK+ihI5+e8VoYfLqsqaJE6ZyFbAYK2v+KqUj7YOOSmmxA9V2zQuEE9dpV8=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB4AzTXlZerE2v7/Ddv2PX3441QCJcjmCtHgGL+/aNwq5dX8jKtNz9IMUa1Wui1rXYF+npugXtLCPABO/UijQRk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'sha1': '9bf1c4ba967dd24077be6e5b87edda4ca87a6e14', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_quiesce']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-15_14:54:08', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-15T15:22:46.217 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-15T15:22:46.222 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-15T15:22:46.274 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2024-04-15T15:22:46.306 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2024-04-15T15:22:46.399 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:46.399 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-04-15T15:22:46.444 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-15T15:22:46.444 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-15 15:22:35.199136205 +0000 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-15 15:19:58.362825911 +0000 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-15 15:19:58.362825911 +0000 2024-04-15T15:22:46.527 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-15 15:19:58.362825911 +0000 2024-04-15T15:22:46.528 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-15T15:22:46.659 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-15T15:22:46.659 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-15T15:22:46.659 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000467935 s, 1.1 MB/s 2024-04-15T15:22:46.663 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-15T15:22:46.718 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-04-15T15:22:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-15T15:22:46.769 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:22:46.769 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-15T15:22:46.769 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:22:46.769 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:22:46.769 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-15 15:22:35.200136185 +0000 2024-04-15T15:22:46.770 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-15 15:19:58.949814277 +0000 2024-04-15T15:22:46.770 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-15 15:19:58.949814277 +0000 2024-04-15T15:22:46.770 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-15 15:19:58.949814277 +0000 2024-04-15T15:22:46.770 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-15T15:22:46.857 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-15T15:22:46.857 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-15T15:22:46.857 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000318353 s, 1.6 MB/s 2024-04-15T15:22:46.860 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-15T15:22:46.918 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-04-15T15:22:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-15T15:22:47.078 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:22:47.078 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-15T15:22:47.078 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:22:47.078 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:22:47.078 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-15 15:22:35.201136165 +0000 2024-04-15T15:22:47.078 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-15 15:19:59.560802167 +0000 2024-04-15T15:22:47.079 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-15 15:19:59.560802167 +0000 2024-04-15T15:22:47.079 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-15 15:19:59.560802167 +0000 2024-04-15T15:22:47.079 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-15T15:22:47.172 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-15T15:22:47.172 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-15T15:22:47.172 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000335367 s, 1.5 MB/s 2024-04-15T15:22:47.176 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-15T15:22:47.228 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-04-15T15:22:47.273 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-15T15:22:47.273 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:22:47.273 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-15T15:22:47.274 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:22:47.274 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:22:47.274 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-15 15:22:35.202136146 +0000 2024-04-15T15:22:47.274 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-15 15:20:00.145790572 +0000 2024-04-15T15:22:47.274 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-15 15:20:00.145790572 +0000 2024-04-15T15:22:47.274 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-15 15:20:00.145790572 +0000 2024-04-15T15:22:47.274 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-15T15:22:47.406 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-15T15:22:47.406 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-15T15:22:47.406 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000387838 s, 1.3 MB/s 2024-04-15T15:22:47.409 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-15T15:22:47.461 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:22:47.462 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:22:47.462 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-04-15T15:22:47.522 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-15T15:22:47.522 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-04-15T15:22:47.573 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-15T15:22:47.574 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:22:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-15T15:22:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:22:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:22:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-15 15:22:23.340948059 +0000 2024-04-15T15:22:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-15 15:19:58.392498898 +0000 2024-04-15T15:22:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-15 15:19:58.392498898 +0000 2024-04-15T15:22:47.574 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-15 15:19:58.392498898 +0000 2024-04-15T15:22:47.575 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-15T15:22:47.664 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-15T15:22:47.664 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-15T15:22:47.664 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000349652 s, 1.5 MB/s 2024-04-15T15:22:47.668 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-15T15:22:47.722 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-04-15T15:22:47.771 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-15T15:22:47.771 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:22:47.771 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-15T15:22:47.772 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:22:47.772 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:22:47.772 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-15 15:22:23.341948042 +0000 2024-04-15T15:22:47.772 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-15 15:19:58.992489055 +0000 2024-04-15T15:22:47.772 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-15 15:19:58.992489055 +0000 2024-04-15T15:22:47.772 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-15 15:19:58.992489055 +0000 2024-04-15T15:22:47.772 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-15T15:22:47.857 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-15T15:22:47.858 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-15T15:22:47.858 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000309254 s, 1.7 MB/s 2024-04-15T15:22:47.862 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-15T15:22:47.918 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-04-15T15:22:47.967 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-15T15:22:47.967 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:22:47.967 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-15T15:22:47.967 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:22:47.967 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:22:47.967 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-15 15:22:23.341948042 +0000 2024-04-15T15:22:47.967 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-15 15:19:59.597479130 +0000 2024-04-15T15:22:47.967 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-15 15:19:59.597479130 +0000 2024-04-15T15:22:47.968 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-15 15:19:59.597479130 +0000 2024-04-15T15:22:47.968 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-15T15:22:48.055 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-15T15:22:48.055 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-15T15:22:48.055 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.00031165 s, 1.6 MB/s 2024-04-15T15:22:48.058 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-15T15:22:48.117 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-04-15T15:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-15T15:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-15T15:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-15T15:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-15T15:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-15T15:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-15 15:22:23.342948026 +0000 2024-04-15T15:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-15 15:20:00.203469188 +0000 2024-04-15T15:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-15 15:20:00.203469188 +0000 2024-04-15T15:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-15 15:20:00.203469188 +0000 2024-04-15T15:22:48.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-15T15:22:48.259 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-15T15:22:48.259 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-15T15:22:48.259 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000308816 s, 1.7 MB/s 2024-04-15T15:22:48.263 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-15T15:22:48.320 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:22:48.321 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-15T15:22:48.321 INFO:tasks.ceph:Generating config... 2024-04-15T15:22:48.322 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-15T15:22:48.322 INFO:tasks.ceph:[client] debug client = 20 2024-04-15T15:22:48.322 INFO:tasks.ceph:[client] debug ms = 1 2024-04-15T15:22:48.322 INFO:tasks.ceph:[client] pid file = 2024-04-15T15:22:48.322 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-15T15:22:48.323 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-15T15:22:48.323 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-15T15:22:48.323 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-15T15:22:48.324 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-15T15:22:48.324 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-15T15:22:48.324 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-15T15:22:48.324 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-15T15:22:48.324 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-15T15:22:48.324 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-15T15:22:48.324 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-15T15:22:48.324 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-15T15:22:48.324 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-15T15:22:48.324 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-15T15:22:48.324 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-15T15:22:48.324 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-15T15:22:48.325 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-15T15:22:48.326 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-15T15:22:48.326 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-15T15:22:48.326 INFO:tasks.ceph:Setting up mon.a... 2024-04-15T15:22:48.326 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-15T15:22:48.470 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.keyring 2024-04-15T15:22:48.478 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-15T15:22:48.612 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:22:48.701 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.150'), ('mon.b', '172.21.15.187'), ('mon.c', '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]')] 2024-04-15T15:22:48.702 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.150,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-15T15:22:48.704 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:48.704 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-15T15:22:48.759 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.150 --add b 172.21.15.187 --addv c '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:22:48.914 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:22:48.914 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: generated fsid 77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4 2024-04-15T15:22:48.914 INFO:teuthology.orchestra.run.smithi150.stdout:setting min_mon_release = quincy 2024-04-15T15:22:48.914 INFO:teuthology.orchestra.run.smithi150.stdout:epoch 0 2024-04-15T15:22:48.914 INFO:teuthology.orchestra.run.smithi150.stdout:fsid 77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4 2024-04-15T15:22:48.914 INFO:teuthology.orchestra.run.smithi150.stdout:last_changed 2024-04-15T15:22:48.913616+0000 2024-04-15T15:22:48.914 INFO:teuthology.orchestra.run.smithi150.stdout:created 2024-04-15T15:22:48.913616+0000 2024-04-15T15:22:48.915 INFO:teuthology.orchestra.run.smithi150.stdout:min_mon_release 17 (quincy) 2024-04-15T15:22:48.915 INFO:teuthology.orchestra.run.smithi150.stdout:election_strategy: 1 2024-04-15T15:22:48.915 INFO:teuthology.orchestra.run.smithi150.stdout:0: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.a 2024-04-15T15:22:48.915 INFO:teuthology.orchestra.run.smithi150.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-04-15T15:22:48.915 INFO:teuthology.orchestra.run.smithi150.stdout:2: [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] mon.c 2024-04-15T15:22:48.915 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-15T15:22:48.917 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-15T15:22:48.968 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4... 2024-04-15T15:22:48.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-15T15:22:48.974 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-15T15:22:49.137 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-04-15T15:22:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-04-15T15:22:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-04-15T15:22:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-04-15T15:22:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.150,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on skipped message = False 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-04-15T15:22:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = bitmap 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-15T15:22:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:22:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-04-15T15:22:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-04-15T15:22:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: mon down mkfs grace = 300 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-15T15:22:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2024-04-15T15:22:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = True 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = True 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-04-15T15:22:49.149 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-04-15T15:22:49.150 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-04-15T15:22:49.151 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-04-15T15:22:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.150,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on skipped message = False 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.153 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: bdev async discard = True 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: bdev enable discard = True 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore allocator = bitmap 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 1/20 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 1/20 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-15T15:22:49.155 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 4/10 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:22:49.156 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-04-15T15:22:49.157 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.158 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: mon op complaint time = 120 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: client mount timeout = 600 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-04-15T15:22:49.159 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout:[mds] 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds = 20 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds balancer = 20 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug frag = True 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug scatterstat = True 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout: mds op complaint time = 180 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout: mds verify scatter = True 2024-04-15T15:22:49.160 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-04-15T15:22:49.161 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-04-15T15:22:49.161 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-04-15T15:22:49.192 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-15T15:22:49.192 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-15T15:22:49.340 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-15T15:22:49.341 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:49.341 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-15T15:22:49.396 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:49.396 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-15T15:22:49.446 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:22:49.446 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:49.446 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-15T15:22:49.446 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:22:49.573 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:22:49.573 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:22:49.668 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:22:49.668 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:22:49.668 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-15T15:22:49.668 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-15T15:22:49.821 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:22:49.821 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:22:49.918 INFO:tasks.ceph:Setting up mon nodes... 2024-04-15T15:22:49.918 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-15T15:22:49.918 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-15T15:22:50.104 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-15T15:22:50.115 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-15T15:22:50.309 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-15T15:22:50.320 INFO:tasks.ceph:Setting up mds nodes... 2024-04-15T15:22:50.320 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-15T15:22:50.503 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-15T15:22:50.513 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-15T15:22:50.608 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-15T15:22:50.787 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-15T15:22:50.796 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-15T15:22:50.889 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-15T15:22:51.076 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-15T15:22:51.086 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-15T15:22:51.175 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-15T15:22:51.349 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-15T15:22:51.359 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-15T15:22:51.451 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-15T15:22:51.451 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-15T15:22:51.581 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-15T15:22:51.639 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-15T15:22:51.764 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-15T15:22:51.827 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-15T15:22:51.954 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-15T15:22:52.004 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-15T15:22:52.128 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-15T15:22:52.179 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-15T15:22:52.180 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-15T15:22:52.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-15T15:22:52.272 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:22:52.272 INFO:tasks.ceph:role: osd.0 2024-04-15T15:22:52.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:22:52.273 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-15T15:22:52.373 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:22:52.373 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:22:52.373 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:22:52.373 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:22:52.374 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:22:52.374 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:22:52.374 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:22:52.374 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:22:52.374 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:22:52.374 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:22:52.784 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-15T15:22:52.790 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-15T15:22:52.790 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-15T15:22:52.949 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-15T15:22:53.050 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-15T15:22:53.142 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:22:53.142 INFO:tasks.ceph:role: osd.1 2024-04-15T15:22:53.142 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:22:53.143 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-15T15:22:53.234 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:22:53.235 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:22:53.591 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-15T15:22:53.596 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-15T15:22:53.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-15T15:22:53.707 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-15T15:22:53.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-15T15:22:53.860 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:22:53.860 INFO:tasks.ceph:role: osd.2 2024-04-15T15:22:53.861 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:22:53.861 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:22:53.945 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:22:54.338 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-15T15:22:54.344 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-15T15:22:54.344 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-15T15:22:54.492 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-15T15:22:54.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-15T15:22:54.656 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-15T15:22:54.656 INFO:tasks.ceph:role: osd.3 2024-04-15T15:22:54.657 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:22:54.657 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-15T15:22:54.746 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:22:54.746 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:22:54.746 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:22:54.746 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:22:54.746 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:22:54.747 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:22:54.747 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:22:54.747 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:22:54.747 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:22:54.747 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:22:55.157 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-15T15:22:55.164 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-15T15:22:55.164 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-15T15:22:55.315 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-15T15:22:55.413 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:22:55.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:55.559+0000 7f5eb69bb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-15T15:22:55.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:55.560+0000 7f5eb69bb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-15T15:22:55.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:55.560+0000 7f5eb69bb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:22:55.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:55.560+0000 7f5eb69bb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:22:55.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:55.560+0000 7f5eb69bb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-15T15:22:55.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:55.563+0000 7f5eb69bb640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-15T15:22:58.541 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-15T15:22:58.643 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:22:58.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:58.778+0000 7f7c941ac640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-15T15:22:58.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:58.779+0000 7f7c941ac640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-15T15:22:58.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:58.779+0000 7f7c941ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:22:58.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:58.779+0000 7f7c941ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:22:58.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:58.779+0000 7f7c941ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-15T15:22:58.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:22:58.781+0000 7f7c941ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-15T15:23:01.747 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-15T15:23:01.841 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:23:01.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:01.980+0000 7f61e76f9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-15T15:23:01.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:01.981+0000 7f61e76f9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-15T15:23:01.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:01.981+0000 7f61e76f9640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:23:01.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:01.981+0000 7f61e76f9640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:23:01.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:01.981+0000 7f61e76f9640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-15T15:23:01.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:01.983+0000 7f61e76f9640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-15T15:23:04.916 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-15T15:23:05.012 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:23:05.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:05.152+0000 7effb496f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-15T15:23:05.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:05.153+0000 7effb496f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-15T15:23:05.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:05.153+0000 7effb496f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:23:05.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:05.153+0000 7effb496f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:23:05.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:05.153+0000 7effb496f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-15T15:23:05.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:05.155+0000 7effb496f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-15T15:23:08.110 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-15T15:23:08.208 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-15T15:23:08.298 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:23:08.298 INFO:tasks.ceph:role: osd.4 2024-04-15T15:23:08.298 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:23:08.298 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-15T15:23:08.397 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:23:08.397 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:23:08.397 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:23:08.397 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:23:08.397 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:23:08.397 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:23:08.397 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:23:08.397 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:23:08.398 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:23:08.398 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:23:08.816 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-15T15:23:08.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-15T15:23:08.823 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-15T15:23:08.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-15T15:23:09.063 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-15T15:23:09.148 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:23:09.148 INFO:tasks.ceph:role: osd.5 2024-04-15T15:23:09.149 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:23:09.149 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:23:09.243 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:23:09.600 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-15T15:23:09.606 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-15T15:23:09.606 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-15T15:23:09.733 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-15T15:23:09.830 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-15T15:23:09.919 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:23:09.920 INFO:tasks.ceph:role: osd.6 2024-04-15T15:23:09.920 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:23:09.920 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-15T15:23:10.012 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:23:10.013 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:23:10.405 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-15T15:23:10.412 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-15T15:23:10.412 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-15T15:23:10.574 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-15T15:23:10.680 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-15T15:23:10.772 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-15T15:23:10.772 INFO:tasks.ceph:role: osd.7 2024-04-15T15:23:10.772 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:23:10.772 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-15T15:23:10.873 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-15T15:23:10.873 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-15T15:23:10.873 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-15T15:23:10.873 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-15T15:23:10.873 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-15T15:23:10.873 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-15T15:23:10.873 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-15T15:23:10.873 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-15T15:23:10.874 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-15T15:23:10.874 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-15T15:23:11.277 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-15T15:23:11.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-15T15:23:11.286 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-15T15:23:11.405 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-15T15:23:11.493 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:23:11.634 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:11.633+0000 7fd548c15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-15T15:23:11.635 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:11.633+0000 7fd548c15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-15T15:23:11.635 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:11.633+0000 7fd548c15640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:23:11.635 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:11.633+0000 7fd548c15640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:23:11.635 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:11.634+0000 7fd548c15640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-15T15:23:11.638 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:11.637+0000 7fd548c15640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-15T15:23:14.621 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-15T15:23:14.716 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:23:14.851 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:14.850+0000 7f63a9b1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-15T15:23:14.852 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:14.851+0000 7f63a9b1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-15T15:23:14.852 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:14.851+0000 7f63a9b1b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:23:14.852 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:14.851+0000 7f63a9b1b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:23:14.852 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:14.851+0000 7f63a9b1b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-15T15:23:14.854 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:14.854+0000 7f63a9b1b640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-15T15:23:16.218 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:23:16.226 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:23:17.802 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-15T15:23:17.899 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:23:18.040 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:18.038+0000 7f25dcb06640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-15T15:23:18.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:18.039+0000 7f25dcb06640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-15T15:23:18.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:18.039+0000 7f25dcb06640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:23:18.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:18.040+0000 7f25dcb06640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:23:18.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:18.040+0000 7f25dcb06640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-15T15:23:18.043 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:18.042+0000 7f25dcb06640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-15T15:23:21.006 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-15T15:23:21.102 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:23:21.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:21.242+0000 7fcbe5c11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-15T15:23:21.245 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:21.243+0000 7fcbe5c11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-15T15:23:21.245 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:21.243+0000 7fcbe5c11640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:23:21.245 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:21.243+0000 7fcbe5c11640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:23:21.245 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:21.243+0000 7fcbe5c11640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-15T15:23:21.247 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:21.245+0000 7fcbe5c11640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-15T15:23:24.195 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-15T15:23:24.294 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-15T15:23:24.295 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:24.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-15T15:23:24.400 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:24.400 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-15T15:23:24.494 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:24.494 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-15T15:23:24.585 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:24.586 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-15T15:23:24.723 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:24.723 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-15T15:23:24.809 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:24.809 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-15T15:23:24.903 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:24.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-15T15:23:24.998 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:24.998 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-15T15:23:25.098 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:25.098 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-15T15:23:25.193 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:25.193 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-15T15:23:25.285 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:25.285 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-15T15:23:25.378 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:25.378 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-15T15:23:25.471 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:25.471 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-15T15:23:25.566 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:25.566 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-15T15:23:25.660 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:25.660 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-15T15:23:25.718 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:25.718 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-15T15:23:25.771 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:25.771 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-15T15:23:25.828 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:25.829 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-15T15:23:25.879 INFO:tasks.ceph:Adding keys to all mons... 2024-04-15T15:23:25.879 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-15T15:23:25.887 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-15T15:23:25.974 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2024-04-15T15:23:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBKRh1md7UyBhAAlb270hfgvbw7PyNlE4iJyQ== 2024-04-15T15:23:25.974 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBKRh1mpj3+HRAAEh8ivBX0TX5YWjk24rgdsQ== 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBKRh1mLyTjLhAA932wttbBinyuX0HaTQs1iA== 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBPRh1mhJRsIRAAzN77WIEMVi/OrEPGuj+3Zg== 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBSRh1mZmF4LhAA2ikl51iHw8oUr3U2kXIVZQ== 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBVRh1mzaaFOhAAUeYLxRNUQOdbZsE7D4GYjQ== 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBZRh1mLMEsCRAADKns6IJ7E6eHoGjlGRRZ1A== 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2024-04-15T15:23:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBKRh1mFIJlEhAA43SJvZPpyghWYT16hxEnIw== 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBLRh1mjOuJBBAAeGWYhd6bfV40OMnCvrcapQ== 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBLRh1mrwjUFBAAtMnGVZa/OdsIXd1N/dI2TQ== 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBfRh1my6zIJRAAJb2eCGYAs/3ny43n+A/Z6w== 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBiRh1mwPS/MhAAy7mjMXTIoDoULw6U9E+z8Q== 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBmRh1mCKFbAhAAPRINFqJje8M4SfTJe13S0A== 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBpRh1maamQDhAAiD+0Z5GY3lCtdDZCQEknBA== 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBLRh1mMhWcIhAALcn3OkkYUWxiZFNe4T1a0A== 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout:[client.2] 2024-04-15T15:23:25.976 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBLRh1mfVuJLRAAMLqOcfYptvjLOA6WhIto/g== 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBLRh1md4XlOBAAVojA1/Gr3hDMDnqtV1d2GA== 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi150.stdout:[client.3] 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBMRh1mA5+cBxAA+C06nPelPPXzUGWAGHi+0w== 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBKRh1md7UyBhAAlb270hfgvbw7PyNlE4iJyQ== 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBKRh1mpj3+HRAAEh8ivBX0TX5YWjk24rgdsQ== 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-04-15T15:23:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBKRh1mLyTjLhAA932wttbBinyuX0HaTQs1iA== 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPRh1mhJRsIRAAzN77WIEMVi/OrEPGuj+3Zg== 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBSRh1mZmF4LhAA2ikl51iHw8oUr3U2kXIVZQ== 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBVRh1mzaaFOhAAUeYLxRNUQOdbZsE7D4GYjQ== 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBZRh1mLMEsCRAADKns6IJ7E6eHoGjlGRRZ1A== 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBKRh1mFIJlEhAA43SJvZPpyghWYT16hxEnIw== 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBLRh1mjOuJBBAAeGWYhd6bfV40OMnCvrcapQ== 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBLRh1mrwjUFBAAtMnGVZa/OdsIXd1N/dI2TQ== 2024-04-15T15:23:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBfRh1my6zIJRAAJb2eCGYAs/3ny43n+A/Z6w== 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBiRh1mwPS/MhAAy7mjMXTIoDoULw6U9E+z8Q== 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBmRh1mCKFbAhAAPRINFqJje8M4SfTJe13S0A== 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBpRh1maamQDhAAiD+0Z5GY3lCtdDZCQEknBA== 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBLRh1mMhWcIhAALcn3OkkYUWxiZFNe4T1a0A== 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout:[client.2] 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBLRh1mfVuJLRAAMLqOcfYptvjLOA6WhIto/g== 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBLRh1md4XlOBAAVojA1/Gr3hDMDnqtV1d2GA== 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout:[client.3] 2024-04-15T15:23:25.979 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBMRh1mA5+cBxAA+C06nPelPPXzUGWAGHi+0w== 2024-04-15T15:23:25.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:23:25.989 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:23:26.130 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:23:26.136 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:23:26.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:23:26.284 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:23:26.427 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:26.432 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:26.563 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:26.568 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:26.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:26.718 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:26.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:26.860 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:26.994 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:23:27.001 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-15T15:23:27.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:23:27.146 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:23:27.292 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:23:27.299 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-15T15:23:27.442 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:27.447 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:27.578 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:27.584 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:27.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:27.722 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:27.860 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:27.865 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-15T15:23:28.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:23:28.008 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:23:28.144 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:23:28.149 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:23:28.291 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:23:28.297 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:23:28.446 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:23:28.451 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-15T15:23:28.599 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-15T15:23:28.599 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-15T15:23:28.688 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:23:28.838 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-15T15:23:28.928 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-15T15:23:29.028 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:23:29.181 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-15T15:23:29.266 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-15T15:23:29.363 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-15T15:23:29.516 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-15T15:23:29.612 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:23:29.620 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-15T15:23:29.672 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-15T15:23:29.673 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-15T15:23:29.673 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-15T15:23:29.678 INFO:tasks.ceph.mon.a:Started 2024-04-15T15:23:29.678 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-15T15:23:29.678 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-15T15:23:29.683 INFO:tasks.ceph.mon.b:Started 2024-04-15T15:23:29.683 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-15T15:23:29.684 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-15T15:23:29.688 INFO:tasks.ceph.mon.c:Started 2024-04-15T15:23:29.688 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-15T15:23:29.688 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-15T15:23:29.688 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-15T15:23:29.693 INFO:tasks.ceph.mgr.y:Started 2024-04-15T15:23:29.693 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-15T15:23:29.693 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-15T15:23:29.697 INFO:tasks.ceph.mgr.x:Started 2024-04-15T15:23:29.697 INFO:tasks.ceph:Setting crush tunables to default 2024-04-15T15:23:29.697 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd crush tunables default 2024-04-15T15:23:29.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.874+0000 7f1784400640 1 Processor -- start 2024-04-15T15:23:29.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.874+0000 7f1784400640 1 -- start start 2024-04-15T15:23:29.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.875+0000 7f1784400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:29.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.875+0000 7f1784400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:29.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.875+0000 7f1784400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:29.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.875+0000 7f1784400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f177c15f0a0 con 0x7f177c15ecc0 2024-04-15T15:23:29.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.875+0000 7f1784400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f177c0ba280 con 0x7f177c0b3720 2024-04-15T15:23:29.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.875+0000 7f1784400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0ba4d0 con 0x7f177c0d3270 2024-04-15T15:23:29.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:29.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43228/0 (socket says 172.21.15.150:43228) 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781a00640 1 -- 172.21.15.150:0/1324640180 learned_addr learned my addr 172.21.15.150:0/1324640180 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781a00640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 msgr2=0x7f177c0d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781a00640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 msgr2=0x7f177c0d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781a00640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781a00640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781000640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781000640 1 -- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 msgr2=0x7f177c0b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781000640 1 -- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 msgr2=0x7f177c0b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781000640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1781000640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:23:29.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1782400640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 msgr2=0x7f177c0b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-15T15:23:29.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1782400640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 msgr2=0x7f177c0b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:23:29.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:23:29.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:29.876+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-15T15:23:30.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.077+0000 7f1781a00640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.077+0000 7f1781000640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1781a00640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 msgr2=0x7f177c0d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-15T15:23:30.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1781a00640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 msgr2=0x7f177c0d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:23:30.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1781a00640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1781a00640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1781000640 1 -- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 msgr2=0x7f177c0b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1781000640 1 -- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 msgr2=0x7f177c0b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1781000640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1782400640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 msgr2=0x7f177c0b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1781000640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1782400640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 msgr2=0x7f177c0b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-15T15:23:30.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.078+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-15T15:23:30.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.479+0000 7f1781a00640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.479+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.187:3300/0 conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.479+0000 7f1781000640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.480+0000 7f1782400640 1 -- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 msgr2=0x7f177c0b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:30.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.480+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.480+0000 7f1782400640 1 -- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 msgr2=0x7f177c0d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:30.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.480+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.480+0000 7f1782400640 1 -- 172.21.15.150:0/1324640180 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c0ba7b0 con 0x7f177c0b3720 2024-04-15T15:23:30.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.483+0000 7f1782400640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0b3720 0x7f177c0b9b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f177c06ce40 tx=0x7f177402c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162c5c329c00d67e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:30.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.483+0000 7f177be00640 1 -- 172.21.15.150:0/1324640180 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774038070 con 0x7f177c0b3720 2024-04-15T15:23:30.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.484+0000 7f177be00640 1 -- 172.21.15.150:0/1324640180 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1774031710 con 0x7f177c0b3720 2024-04-15T15:23:30.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.484+0000 7f177be00640 1 -- 172.21.15.150:0/1324640180 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774031b40 con 0x7f177c0b3720 2024-04-15T15:23:30.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.485+0000 7f1784400640 1 -- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0b3720 msgr2=0x7f177c0b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:30.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.485+0000 7f1784400640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0b3720 0x7f177c0b9b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f177c06ce40 tx=0x7f177402c4c0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.486+0000 7f1784400640 1 -- 172.21.15.150:0/1324640180 shutdown_connections 2024-04-15T15:23:30.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.486+0000 7f1784400640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0b3720 0x7f177c0b9b30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.486+0000 7f1784400640 1 --2- 172.21.15.150:0/1324640180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0d3270 0x7f177c0b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.486+0000 7f1784400640 1 --2- 172.21.15.150:0/1324640180 >> v2:172.21.15.150:3300/0 conn(0x7f177c15ecc0 0x7f177c0d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.486+0000 7f1784400640 1 -- 172.21.15.150:0/1324640180 >> 172.21.15.150:0/1324640180 conn(0x7f177c073480 msgr2=0x7f177c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:30.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.486+0000 7f1784400640 1 -- 172.21.15.150:0/1324640180 shutdown_connections 2024-04-15T15:23:30.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.486+0000 7f1784400640 1 -- 172.21.15.150:0/1324640180 wait complete. 2024-04-15T15:23:30.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.489+0000 7f1784400640 1 Processor -- start 2024-04-15T15:23:30.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.490+0000 7f1784400640 1 -- start start 2024-04-15T15:23:30.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.491+0000 7f1784400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0b3720 0x7f177c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:30.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.491+0000 7f1784400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c0d3270 0x7f177c160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:30.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.491+0000 7f1784400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c15ecc0 0x7f177c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:30.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.491+0000 7f1784400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0b3c70 con 0x7f177c0d3270 2024-04-15T15:23:30.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.491+0000 7f1784400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0b90d0 con 0x7f177c15ecc0 2024-04-15T15:23:30.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.491+0000 7f1784400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0b3df0 con 0x7f177c0b3720 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.492+0000 7f1781a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0b3720 0x7f177c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.492+0000 7f1782400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c15ecc0 0x7f177c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.492+0000 7f1781000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c0d3270 0x7f177c160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.492+0000 7f1782400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c15ecc0 0x7f177c0c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:48974/0 (socket says 172.21.15.150:48974) 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.492+0000 7f1782400640 1 -- 172.21.15.150:0/2530847333 learned_addr learned my addr 172.21.15.150:0/2530847333 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.492+0000 7f1781000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c0d3270 0x7f177c160cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43268/0 (socket says 172.21.15.150:43268) 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1781000640 1 -- 172.21.15.150:0/2530847333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0b3720 msgr2=0x7f177c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1781000640 1 --2- 172.21.15.150:0/2530847333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0b3720 0x7f177c15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1781000640 1 -- 172.21.15.150:0/2530847333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c15ecc0 msgr2=0x7f177c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1781000640 1 --2- 172.21.15.150:0/2530847333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c15ecc0 0x7f177c0c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1781000640 1 -- 172.21.15.150:0/2530847333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c15f800 con 0x7f177c0d3270 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1782400640 1 --2- 172.21.15.150:0/2530847333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c15ecc0 0x7f177c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:23:30.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1781000640 1 --2- 172.21.15.150:0/2530847333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c0d3270 0x7f177c160cc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1768003f90 tx=0x7f176800d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:30.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1779600640 1 -- 172.21.15.150:0/2530847333 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768019070 con 0x7f177c0d3270 2024-04-15T15:23:30.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1779600640 1 -- 172.21.15.150:0/2530847333 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1768014680 con 0x7f177c0d3270 2024-04-15T15:23:30.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1779600640 1 -- 172.21.15.150:0/2530847333 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768014a30 con 0x7f177c0d3270 2024-04-15T15:23:30.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.493+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c15fae0 con 0x7f177c0d3270 2024-04-15T15:23:30.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.494+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1600a0 con 0x7f177c0d3270 2024-04-15T15:23:30.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.494+0000 7f1779600640 1 -- 172.21.15.150:0/2530847333 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f176801c070 con 0x7f177c0d3270 2024-04-15T15:23:30.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.494+0000 7f1779600640 1 -- 172.21.15.150:0/2530847333 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1768015790 con 0x7f177c0d3270 2024-04-15T15:23:30.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.495+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c068c80 con 0x7f177c0d3270 2024-04-15T15:23:30.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.498+0000 7f1779600640 1 -- 172.21.15.150:0/2530847333 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1768014c20 con 0x7f177c0d3270 2024-04-15T15:23:30.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.590+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f177c0b3500 con 0x7f177c0d3270 2024-04-15T15:23:30.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.594+0000 7f1779600640 1 -- 172.21.15.150:0/2530847333 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f1768014e00 con 0x7f177c0d3270 2024-04-15T15:23:30.595 INFO:teuthology.orchestra.run.smithi150.stderr:adjusted tunables profile to default 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.595+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c0d3270 msgr2=0x7f177c160cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.595+0000 7f1784400640 1 --2- 172.21.15.150:0/2530847333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c0d3270 0x7f177c160cc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1768003f90 tx=0x7f176800d930 comp rx=0 tx=0).stop 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.596+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 shutdown_connections 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.596+0000 7f1784400640 1 --2- 172.21.15.150:0/2530847333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c15ecc0 0x7f177c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.596+0000 7f1784400640 1 --2- 172.21.15.150:0/2530847333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c0d3270 0x7f177c160cc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.596+0000 7f1784400640 1 --2- 172.21.15.150:0/2530847333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0b3720 0x7f177c15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.596+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 >> 172.21.15.150:0/2530847333 conn(0x7f177c073480 msgr2=0x7f177c0d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.596+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 shutdown_connections 2024-04-15T15:23:30.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:30.596+0000 7f1784400640 1 -- 172.21.15.150:0/2530847333 wait complete. 2024-04-15T15:23:30.616 INFO:tasks.ceph:check_enable_crimson: False 2024-04-15T15:23:30.616 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-15T15:23:30.616 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:30.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-15T15:23:30.700 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:30.700 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-15T15:23:30.773 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:30.771+0000 7fa6781b8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-15T15:23:30.823 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:30.822+0000 7f70d56da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-15T15:23:30.831 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:30.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-15T15:23:30.841 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:30.839+0000 7fa6781b8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-15T15:23:30.894 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:30.892+0000 7f70d56da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-15T15:23:30.913 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:23:30.913 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-15T15:23:31.039 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:31.039 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-15T15:23:31.130 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:31.130 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-15T15:23:31.214 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:31.214 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-15T15:23:31.344 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:23:31.344 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-15T15:23:31.465 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 9eade02f-52ed-4fe2-9498-a568d26e296e 0 2024-04-15T15:23:31.480 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:31.478+0000 7fa6781b8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-15T15:23:31.508 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:31.507+0000 7f70d56da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-15T15:23:31.549 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:31.546+0000 7fa6781b8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-15T15:23:31.574 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:31.572+0000 7f70d56da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-15T15:23:31.602 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:31.600+0000 7fa6781b8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-15T15:23:31.625 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:31.624+0000 7f70d56da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-15T15:23:31.661 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.659+0000 7f1783200640 1 Processor -- start 2024-04-15T15:23:31.661 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.659+0000 7f1783200640 1 -- start start 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.660+0000 7f1783200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c15eb40 0x7f177c162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.660+0000 7f1783200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f177c0ba460 0x7f177c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.660+0000 7f1783200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177c163400 con 0x7f177c0d1480 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.660+0000 7f1783200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f177c162e40 con 0x7f177c0ba460 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.660+0000 7f1783200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f177c163120 con 0x7f177c15eb40 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c15eb40 0x7f177c162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f177c0ba460 0x7f177c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c15eb40 0x7f177c162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58138/0 (socket says 172.21.15.187:58138) 2024-04-15T15:23:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781e00640 1 -- 172.21.15.187:0/1012686595 learned_addr learned my addr 172.21.15.187:0/1012686595 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:31.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781e00640 1 -- 172.21.15.187:0/1012686595 >> v2:172.21.15.187:3300/0 conn(0x7f177c0ba460 msgr2=0x7f177c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:31.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781e00640 1 --2- 172.21.15.187:0/1012686595 >> v2:172.21.15.187:3300/0 conn(0x7f177c0ba460 0x7f177c0d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781e00640 1 -- 172.21.15.187:0/1012686595 >> v1:172.21.15.187:6789/0 conn(0x7f177c0d1480 legacy=0x7f177c0b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:31.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781e00640 1 -- 172.21.15.187:0/1012686595 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c0d2bc0 con 0x7f177c15eb40 2024-04-15T15:23:31.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.661+0000 7f1781400640 1 --2- 172.21.15.187:0/1012686595 >> v2:172.21.15.187:3300/0 conn(0x7f177c0ba460 0x7f177c0d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:23:31.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.662+0000 7f1781e00640 1 --2- 172.21.15.187:0/1012686595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c15eb40 0x7f177c162510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1770009870 tx=0x7f177002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b48e1c8f69b47660 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1780a00640 1 -- 172.21.15.187:0/1012686595 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770035020 con 0x7f177c15eb40 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1780a00640 1 -- 172.21.15.187:0/1012686595 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1770031640 con 0x7f177c15eb40 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1780a00640 1 -- 172.21.15.187:0/1012686595 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177002d040 con 0x7f177c15eb40 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1783200640 1 -- 172.21.15.187:0/1012686595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c15eb40 msgr2=0x7f177c162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1783200640 1 --2- 172.21.15.187:0/1012686595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c15eb40 0x7f177c162510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1770009870 tx=0x7f177002b8f0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1783200640 1 -- 172.21.15.187:0/1012686595 shutdown_connections 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1783200640 1 --2- 172.21.15.187:0/1012686595 >> v2:172.21.15.187:3300/0 conn(0x7f177c0ba460 0x7f177c0d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1783200640 1 --2- 172.21.15.187:0/1012686595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c15eb40 0x7f177c162510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1783200640 1 -- 172.21.15.187:0/1012686595 >> 172.21.15.187:0/1012686595 conn(0x7f177c06d3d0 msgr2=0x7f177c073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1783200640 1 -- 172.21.15.187:0/1012686595 shutdown_connections 2024-04-15T15:23:31.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.663+0000 7f1783200640 1 -- 172.21.15.187:0/1012686595 wait complete. 2024-04-15T15:23:31.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.665+0000 7f1783200640 1 Processor -- start 2024-04-15T15:23:31.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.665+0000 7f1783200640 1 -- start start 2024-04-15T15:23:31.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1783200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0ba460 0x7f177c14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1783200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0d1480 0x7f177c161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1783200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c15eb40 0x7f177c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1783200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0d43d0 con 0x7f177c15eb40 2024-04-15T15:23:31.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1783200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0b7860 con 0x7f177c0d1480 2024-04-15T15:23:31.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1783200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0d4550 con 0x7f177c0ba460 2024-04-15T15:23:31.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1781e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0ba460 0x7f177c14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.669 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1781400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0d1480 0x7f177c161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1781e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0ba460 0x7f177c14c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58146/0 (socket says 172.21.15.187:58146) 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.666+0000 7f1781e00640 1 -- 172.21.15.187:0/1306376305 learned_addr learned my addr 172.21.15.187:0/1306376305 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1781e00640 1 -- 172.21.15.187:0/1306376305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0d1480 msgr2=0x7f177c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1782800640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c15eb40 0x7f177c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1781e00640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0d1480 0x7f177c161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1781e00640 1 -- 172.21.15.187:0/1306376305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c15eb40 msgr2=0x7f177c14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1781e00640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c15eb40 0x7f177c14e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1781e00640 1 -- 172.21.15.187:0/1306376305 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c161fc0 con 0x7f177c0ba460 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1781400640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0d1480 0x7f177c161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1782800640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c15eb40 0x7f177c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:31.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1781e00640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0ba460 0x7f177c14c280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1770000c00 tx=0x7f17700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:31.671 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:31.667+0000 7fa6781b8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-15T15:23:31.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1776000640 1 -- 172.21.15.187:0/1306376305 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770040070 con 0x7f177c0ba460 2024-04-15T15:23:31.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1776000640 1 -- 172.21.15.187:0/1306376305 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17700399f0 con 0x7f177c0ba460 2024-04-15T15:23:31.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1776000640 1 -- 172.21.15.187:0/1306376305 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770035040 con 0x7f177c0ba460 2024-04-15T15:23:31.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.667+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c15efc0 con 0x7f177c0ba460 2024-04-15T15:23:31.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.668+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c15f580 con 0x7f177c0ba460 2024-04-15T15:23:31.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.668+0000 7f1776000640 1 -- 172.21.15.187:0/1306376305 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f177004b060 con 0x7f177c0ba460 2024-04-15T15:23:31.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.668+0000 7f1776000640 1 -- 172.21.15.187:0/1306376305 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f177003a420 con 0x7f177c0ba460 2024-04-15T15:23:31.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.669+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c0755c0 con 0x7f177c0ba460 2024-04-15T15:23:31.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.671+0000 7f1776000640 1 -- 172.21.15.187:0/1306376305 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f177004d970 con 0x7f177c0ba460 2024-04-15T15:23:31.701 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:31.700+0000 7f70d56da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-15T15:23:31.722 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:31.720+0000 7fa6781b8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-15T15:23:31.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.750+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9eade02f-52ed-4fe2-9498-a568d26e296e", "id": 0} v 0) -- 0x7f177c0d0bb0 con 0x7f177c0ba460 2024-04-15T15:23:31.753 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:31.752+0000 7f70d56da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-15T15:23:31.761 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.759+0000 7f1776000640 1 -- 172.21.15.187:0/1306376305 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9eade02f-52ed-4fe2-9498-a568d26e296e", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f177004db50 con 0x7f177c0ba460 2024-04-15T15:23:31.762 INFO:teuthology.orchestra.run.smithi187.stdout:0 2024-04-15T15:23:31.763 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0ba460 msgr2=0x7f177c14c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:31.763 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0ba460 0x7f177c14c280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1770000c00 tx=0x7f17700046c0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.763 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 shutdown_connections 2024-04-15T15:23:31.763 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f177c15eb40 0x7f177c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.763 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f177c0d1480 0x7f177c161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.763 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 --2- 172.21.15.187:0/1306376305 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f177c0ba460 0x7f177c14c280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.763 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 >> 172.21.15.187:0/1306376305 conn(0x7f177c06d3d0 msgr2=0x7f177c0c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:31.763 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 shutdown_connections 2024-04-15T15:23:31.764 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.761+0000 7f1783200640 1 -- 172.21.15.187:0/1306376305 wait complete. 2024-04-15T15:23:31.779 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 7dd7200c-18bc-4ded-9d75-f9f7be236c75 1 2024-04-15T15:23:31.862 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:31.860+0000 7fa6781b8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-15T15:23:31.887 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:31.885+0000 7f70d56da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-15T15:23:31.946 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:31.944+0000 7fa6781b8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-15T15:23:31.967 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:31.966+0000 7f70d56da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-15T15:23:31.969 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.967+0000 7f25d7e00640 1 Processor -- start 2024-04-15T15:23:31.969 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.967+0000 7f25d7e00640 1 -- start start 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.968+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d8162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.968+0000 7f25d7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25d80ba460 0x7f25d80d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.968+0000 7f25d7e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d8163400 con 0x7f25d80d1480 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d7e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f25d8162e40 con 0x7f25d80ba460 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25d8163120 con 0x7f25d815eb40 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25d80ba460 0x7f25d80d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d8162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d8162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58156/0 (socket says 172.21.15.187:58156) 2024-04-15T15:23:31.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6a00640 1 -- 172.21.15.187:0/2940986290 learned_addr learned my addr 172.21.15.187:0/2940986290 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:31.972 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6a00640 1 -- 172.21.15.187:0/2940986290 >> v2:172.21.15.187:3300/0 conn(0x7f25d80ba460 msgr2=0x7f25d80d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:31.972 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6a00640 1 --2- 172.21.15.187:0/2940986290 >> v2:172.21.15.187:3300/0 conn(0x7f25d80ba460 0x7f25d80d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.972 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6a00640 1 -- 172.21.15.187:0/2940986290 >> v1:172.21.15.150:6789/0 conn(0x7f25d80d1480 legacy=0x7f25d80b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:31.972 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6a00640 1 -- 172.21.15.187:0/2940986290 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d80d2bc0 con 0x7f25d815eb40 2024-04-15T15:23:31.972 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.969+0000 7f25d6000640 1 --2- 172.21.15.187:0/2940986290 >> v2:172.21.15.187:3300/0 conn(0x7f25d80ba460 0x7f25d80d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:23:31.973 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.970+0000 7f25d6a00640 1 --2- 172.21.15.187:0/2940986290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d8162510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009e50 tx=0x7f25d002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75067ea8a6b2967d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:31.973 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d5600640 1 -- 172.21.15.187:0/2940986290 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d002c9f0 con 0x7f25d815eb40 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d5600640 1 -- 172.21.15.187:0/2940986290 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d002cb90 con 0x7f25d815eb40 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d5600640 1 -- 172.21.15.187:0/2940986290 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00343f0 con 0x7f25d815eb40 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d7e00640 1 -- 172.21.15.187:0/2940986290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 msgr2=0x7f25d8162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d7e00640 1 --2- 172.21.15.187:0/2940986290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d8162510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009e50 tx=0x7f25d002be50 comp rx=0 tx=0).stop 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d7e00640 1 -- 172.21.15.187:0/2940986290 shutdown_connections 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d7e00640 1 --2- 172.21.15.187:0/2940986290 >> v2:172.21.15.187:3300/0 conn(0x7f25d80ba460 0x7f25d80d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d7e00640 1 --2- 172.21.15.187:0/2940986290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d8162510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.971+0000 7f25d7e00640 1 -- 172.21.15.187:0/2940986290 >> 172.21.15.187:0/2940986290 conn(0x7f25d806d3d0 msgr2=0x7f25d8073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.972+0000 7f25d7e00640 1 -- 172.21.15.187:0/2940986290 shutdown_connections 2024-04-15T15:23:31.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.972+0000 7f25d7e00640 1 -- 172.21.15.187:0/2940986290 wait complete. 2024-04-15T15:23:31.975 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.973+0000 7f25d7e00640 1 Processor -- start 2024-04-15T15:23:31.975 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.973+0000 7f25d7e00640 1 -- start start 2024-04-15T15:23:31.977 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.974+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d80ba460 0x7f25d8153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.977 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.974+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d80d1480 0x7f25d80b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.977 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.974+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:31.977 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.974+0000 7f25d7e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25d80d43d0 con 0x7f25d80ba460 2024-04-15T15:23:31.977 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.974+0000 7f25d7e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25d80b7860 con 0x7f25d80d1480 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.974+0000 7f25d7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25d80d4550 con 0x7f25d815eb40 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d6000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d80d1480 0x7f25d80b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d815d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58170/0 (socket says 172.21.15.187:58170) 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 -- 172.21.15.187:0/3587526500 learned_addr learned my addr 172.21.15.187:0/3587526500 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d6000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d80d1480 0x7f25d80b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46110/0 (socket says 172.21.15.187:46110) 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d6a00640 1 --2- 172.21.15.187:0/3587526500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d80ba460 0x7f25d8153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 -- 172.21.15.187:0/3587526500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d80d1480 msgr2=0x7f25d80b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 --2- 172.21.15.187:0/3587526500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d80d1480 0x7f25d80b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 -- 172.21.15.187:0/3587526500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d80ba460 msgr2=0x7f25d8153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 --2- 172.21.15.187:0/3587526500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d80ba460 0x7f25d8153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 -- 172.21.15.187:0/3587526500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d80b6040 con 0x7f25d815eb40 2024-04-15T15:23:31.978 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.975+0000 7f25d7400640 1 --2- 172.21.15.187:0/3587526500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d815d080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f25c800c410 tx=0x7f25c800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:31.979 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.976+0000 7f25cea00640 1 -- 172.21.15.187:0/3587526500 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8011790 con 0x7f25d815eb40 2024-04-15T15:23:31.979 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.976+0000 7f25cea00640 1 -- 172.21.15.187:0/3587526500 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c8017070 con 0x7f25d815eb40 2024-04-15T15:23:31.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.976+0000 7f25cea00640 1 -- 172.21.15.187:0/3587526500 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8011b70 con 0x7f25d815eb40 2024-04-15T15:23:31.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.976+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d80cb930 con 0x7f25d815eb40 2024-04-15T15:23:31.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.976+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d80cbef0 con 0x7f25d815eb40 2024-04-15T15:23:31.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.976+0000 7f25cea00640 1 -- 172.21.15.187:0/3587526500 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f25c8011d10 con 0x7f25d815eb40 2024-04-15T15:23:31.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.976+0000 7f25cea00640 1 -- 172.21.15.187:0/3587526500 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f25c801c460 con 0x7f25d815eb40 2024-04-15T15:23:31.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.977+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d80cc0f0 con 0x7f25d815eb40 2024-04-15T15:23:31.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:31.979+0000 7f25cea00640 1 -- 172.21.15.187:0/3587526500 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f25d80cc0f0 con 0x7f25d815eb40 2024-04-15T15:23:32.004 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:32.002+0000 7fa6781b8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-15T15:23:32.023 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:32.022+0000 7f70d56da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-15T15:23:32.061 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.059+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7dd7200c-18bc-4ded-9d75-f9f7be236c75", "id": 1} v 0) -- 0x7f25d80755c0 con 0x7f25d815eb40 2024-04-15T15:23:32.065 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:32.063+0000 7fa6781b8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-15T15:23:32.068 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.067+0000 7f25cea00640 1 -- 172.21.15.187:0/3587526500 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7dd7200c-18bc-4ded-9d75-f9f7be236c75", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f25c8021020 con 0x7f25d815eb40 2024-04-15T15:23:32.069 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-15T15:23:32.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 msgr2=0x7f25d815d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 --2- 172.21.15.187:0/3587526500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d815d080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f25c800c410 tx=0x7f25c800c8d0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 shutdown_connections 2024-04-15T15:23:32.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 --2- 172.21.15.187:0/3587526500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25d815eb40 0x7f25d815d080 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 --2- 172.21.15.187:0/3587526500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d80d1480 0x7f25d80b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 --2- 172.21.15.187:0/3587526500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d80ba460 0x7f25d8153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 >> 172.21.15.187:0/3587526500 conn(0x7f25d806d3d0 msgr2=0x7f25d80b8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:32.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 shutdown_connections 2024-04-15T15:23:32.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.068+0000 7f25d7e00640 1 -- 172.21.15.187:0/3587526500 wait complete. 2024-04-15T15:23:32.087 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:32.085+0000 7f70d56da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-15T15:23:32.088 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new ddd19ed9-014c-4e0a-928c-21833f47ae17 2 2024-04-15T15:23:32.124 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:32.122+0000 7fa6781b8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-15T15:23:32.144 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:32.143+0000 7f70d56da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-15T15:23:32.275 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.273+0000 7fc636e00640 1 Processor -- start 2024-04-15T15:23:32.275 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.273+0000 7fc636e00640 1 -- start start 2024-04-15T15:23:32.276 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.274+0000 7fc636e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6301698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.277 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.274+0000 7fc636e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc630163450 con 0x7fc6300ba3e0 2024-04-15T15:23:32.277 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.274+0000 7fc636e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc630162e90 con 0x7fc63015e9a0 2024-04-15T15:23:32.277 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.274+0000 7fc636e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc630163170 con 0x7fc6300cfc00 2024-04-15T15:23:32.277 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.275+0000 7fc635a00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fc63015e9a0 0x7fc630162590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:36012/0 (socket says 172.21.15.187:36012) 2024-04-15T15:23:32.277 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.275+0000 7fc635a00640 1 -- 172.21.15.187:0/2773723203 learned_addr learned my addr 172.21.15.187:0/2773723203 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-15T15:23:32.277 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.275+0000 7fc636400640 1 --2- 172.21.15.187:0/2773723203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6301698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.277 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.275+0000 7fc636400640 1 -- 172.21.15.187:0/2773723203 >> v1:172.21.15.187:6789/0 conn(0x7fc63015e9a0 legacy=0x7fc630162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.277 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.275+0000 7fc62fe00640 1 -- 172.21.15.187:0/2773723203 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1938603644 0 0) 0x7fc630162e90 con 0x7fc63015e9a0 2024-04-15T15:23:32.278 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.275+0000 7fc636400640 1 -- 172.21.15.187:0/2773723203 >> v1:172.21.15.150:6789/0 conn(0x7fc6300ba3e0 legacy=0x7fc6300cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:32.278 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.275+0000 7fc636400640 1 -- 172.21.15.187:0/2773723203 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6300d2920 con 0x7fc6300cfc00 2024-04-15T15:23:32.279 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.277+0000 7fc636400640 1 --2- 172.21.15.187:0/2773723203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6301698c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc628009dc0 tx=0x7fc62802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a69b4fb9a2d5cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:32.279 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.277+0000 7fc62fe00640 1 -- 172.21.15.187:0/2773723203 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628039070 con 0x7fc6300cfc00 2024-04-15T15:23:32.279 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.277+0000 7fc62fe00640 1 -- 172.21.15.187:0/2773723203 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62802ccd0 con 0x7fc6300cfc00 2024-04-15T15:23:32.279 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.277+0000 7fc62fe00640 1 -- 172.21.15.187:0/2773723203 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628034660 con 0x7fc6300cfc00 2024-04-15T15:23:32.280 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.278+0000 7fc636e00640 1 -- 172.21.15.187:0/2773723203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 msgr2=0x7fc6301698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.280 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.278+0000 7fc636e00640 1 --2- 172.21.15.187:0/2773723203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6301698c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc628009dc0 tx=0x7fc62802bf20 comp rx=0 tx=0).stop 2024-04-15T15:23:32.280 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.278+0000 7fc636e00640 1 -- 172.21.15.187:0/2773723203 shutdown_connections 2024-04-15T15:23:32.280 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.278+0000 7fc636e00640 1 --2- 172.21.15.187:0/2773723203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6301698c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.280 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.278+0000 7fc636e00640 1 -- 172.21.15.187:0/2773723203 >> 172.21.15.187:0/2773723203 conn(0x7fc63006ce40 msgr2=0x7fc63006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:32.280 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.278+0000 7fc636e00640 1 -- 172.21.15.187:0/2773723203 shutdown_connections 2024-04-15T15:23:32.280 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.278+0000 7fc636e00640 1 -- 172.21.15.187:0/2773723203 wait complete. 2024-04-15T15:23:32.281 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.279+0000 7fc636e00640 1 Processor -- start 2024-04-15T15:23:32.281 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.280+0000 7fc636e00640 1 -- start start 2024-04-15T15:23:32.284 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.280+0000 7fc636e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300ba3e0 0x7fc6301f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.284 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.280+0000 7fc636e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6300c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.284 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.280+0000 7fc636e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc63015e9a0 0x7fc6300cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.284 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.280+0000 7fc636e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc63015ed80 con 0x7fc63015e9a0 2024-04-15T15:23:32.284 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.280+0000 7fc636e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6300d4130 con 0x7fc6300ba3e0 2024-04-15T15:23:32.284 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.280+0000 7fc636e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6300c87c0 con 0x7fc6300cfc00 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300ba3e0 0x7fc6301f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6300c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300ba3e0 0x7fc6301f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46126/0 (socket says 172.21.15.187:46126) 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6300c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58178/0 (socket says 172.21.15.187:58178) 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 -- 172.21.15.187:0/415360491 learned_addr learned my addr 172.21.15.187:0/415360491 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc636400640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc63015e9a0 0x7fc6300cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 -- 172.21.15.187:0/415360491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 msgr2=0x7fc6300c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6300c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 -- 172.21.15.187:0/415360491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc63015e9a0 msgr2=0x7fc6300cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc63015e9a0 0x7fc6300cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 -- 172.21.15.187:0/415360491 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6300c8aa0 con 0x7fc6300ba3e0 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635000640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6300c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:23:32.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.281+0000 7fc635a00640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300ba3e0 0x7fc6301f7920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc62400c430 tx=0x7fc62400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:32.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.282+0000 7fc62d600640 1 -- 172.21.15.187:0/415360491 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6240074e0 con 0x7fc6300ba3e0 2024-04-15T15:23:32.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.282+0000 7fc62d600640 1 -- 172.21.15.187:0/415360491 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6240047e0 con 0x7fc6300ba3e0 2024-04-15T15:23:32.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.282+0000 7fc62d600640 1 -- 172.21.15.187:0/415360491 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624004bd0 con 0x7fc6300ba3e0 2024-04-15T15:23:32.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.282+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6300c8d80 con 0x7fc6300ba3e0 2024-04-15T15:23:32.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.282+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6300c9340 con 0x7fc6300ba3e0 2024-04-15T15:23:32.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.282+0000 7fc62d600640 1 -- 172.21.15.187:0/415360491 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc624011070 con 0x7fc6300ba3e0 2024-04-15T15:23:32.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.282+0000 7fc62d600640 1 -- 172.21.15.187:0/415360491 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc6240137e0 con 0x7fc6300ba3e0 2024-04-15T15:23:32.286 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.283+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6300c9540 con 0x7fc6300ba3e0 2024-04-15T15:23:32.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.286+0000 7fc62d600640 1 -- 172.21.15.187:0/415360491 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc6300c9540 con 0x7fc6300ba3e0 2024-04-15T15:23:32.366 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.364+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ddd19ed9-014c-4e0a-928c-21833f47ae17", "id": 2} v 0) -- 0x7fc6300755c0 con 0x7fc6300ba3e0 2024-04-15T15:23:32.375 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.373+0000 7fc62d600640 1 -- 172.21.15.187:0/415360491 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddd19ed9-014c-4e0a-928c-21833f47ae17", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc624004d70 con 0x7fc6300ba3e0 2024-04-15T15:23:32.375 INFO:teuthology.orchestra.run.smithi187.stdout:2 2024-04-15T15:23:32.377 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.374+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300ba3e0 msgr2=0x7fc6301f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.377 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.374+0000 7fc636e00640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300ba3e0 0x7fc6301f7920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc62400c430 tx=0x7fc62400c8f0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.377 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.374+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 shutdown_connections 2024-04-15T15:23:32.377 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.374+0000 7fc636e00640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc63015e9a0 0x7fc6300cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.378 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.374+0000 7fc636e00640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6300cfc00 0x7fc6300c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.378 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.374+0000 7fc636e00640 1 --2- 172.21.15.187:0/415360491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6300ba3e0 0x7fc6301f7920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.378 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.374+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 >> 172.21.15.187:0/415360491 conn(0x7fc63006ce40 msgr2=0x7fc63015d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:32.378 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.375+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 shutdown_connections 2024-04-15T15:23:32.378 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.375+0000 7fc636e00640 1 -- 172.21.15.187:0/415360491 wait complete. 2024-04-15T15:23:32.393 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new fd9d396e-995d-4e3b-87b2-677311a2d3dc 3 2024-04-15T15:23:32.599 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:32.597+0000 7fa6781b8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-15T15:23:32.602 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:32.600+0000 7f70d56da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-15T15:23:32.622 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.620+0000 7f364b200640 1 Processor -- start 2024-04-15T15:23:32.622 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.620+0000 7f364b200640 1 -- start start 2024-04-15T15:23:32.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f364b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f364415ea60 0x7f36440d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f364b200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364415ee40 con 0x7f364406be60 2024-04-15T15:23:32.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f364b200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3644073f50 con 0x7f3644073900 2024-04-15T15:23:32.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f364b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36440d1880 con 0x7f364415ea60 2024-04-15T15:23:32.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f3649e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f364415ea60 0x7f36440d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f3649e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f364415ea60 0x7f36440d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58180/0 (socket says 172.21.15.187:58180) 2024-04-15T15:23:32.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f3649e00640 1 -- 172.21.15.187:0/3201547514 learned_addr learned my addr 172.21.15.187:0/3201547514 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:32.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f364a800640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f3644073900 0x7f36440cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:36028/0 (socket says 172.21.15.187:36028) 2024-04-15T15:23:32.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f3649e00640 1 -- 172.21.15.187:0/3201547514 >> v1:172.21.15.187:6789/0 conn(0x7f3644073900 legacy=0x7f36440cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f3649e00640 1 -- 172.21.15.187:0/3201547514 >> v1:172.21.15.150:6789/0 conn(0x7f364406be60 legacy=0x7f36440731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:32.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.622+0000 7f3649e00640 1 -- 172.21.15.187:0/3201547514 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440d0f50 con 0x7f364415ea60 2024-04-15T15:23:32.626 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.623+0000 7f3649e00640 1 --2- 172.21.15.187:0/3201547514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f364415ea60 0x7f36440d1d10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f36380098a0 tx=0x7f363802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68db626092c38532 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:32.626 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.624+0000 7f3648a00640 1 -- 172.21.15.187:0/3201547514 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638034070 con 0x7f364415ea60 2024-04-15T15:23:32.626 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.624+0000 7f3648a00640 1 -- 172.21.15.187:0/3201547514 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3638030840 con 0x7f364415ea60 2024-04-15T15:23:32.626 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.624+0000 7f3648a00640 1 -- 172.21.15.187:0/3201547514 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638030c10 con 0x7f364415ea60 2024-04-15T15:23:32.627 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.624+0000 7f364b200640 1 -- 172.21.15.187:0/3201547514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f364415ea60 msgr2=0x7f36440d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.627 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.624+0000 7f364b200640 1 --2- 172.21.15.187:0/3201547514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f364415ea60 0x7f36440d1d10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f36380098a0 tx=0x7f363802b980 comp rx=0 tx=0).stop 2024-04-15T15:23:32.627 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.624+0000 7f364b200640 1 -- 172.21.15.187:0/3201547514 shutdown_connections 2024-04-15T15:23:32.627 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.624+0000 7f364b200640 1 --2- 172.21.15.187:0/3201547514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f364415ea60 0x7f36440d1d10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.627 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.625+0000 7f364b200640 1 -- 172.21.15.187:0/3201547514 >> 172.21.15.187:0/3201547514 conn(0x7f364406ca40 msgr2=0x7f364406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:32.627 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.625+0000 7f364b200640 1 -- 172.21.15.187:0/3201547514 shutdown_connections 2024-04-15T15:23:32.627 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.625+0000 7f364b200640 1 -- 172.21.15.187:0/3201547514 wait complete. 2024-04-15T15:23:32.628 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.626+0000 7f364b200640 1 Processor -- start 2024-04-15T15:23:32.628 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.626+0000 7f364b200640 1 -- start start 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364406be60 0x7f36440b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3644073900 0x7f36440cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364415ea60 0x7f364414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f36440d2250 con 0x7f364406be60 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36440d1510 con 0x7f364415ea60 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36440ce6c0 con 0x7f3644073900 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364a800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364415ea60 0x7f364414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364a800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364415ea60 0x7f364414a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46140/0 (socket says 172.21.15.187:46140) 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f3649400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3644073900 0x7f36440cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.627+0000 7f364a800640 1 -- 172.21.15.187:0/1131764917 learned_addr learned my addr 172.21.15.187:0/1131764917 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f3649e00640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364406be60 0x7f36440b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f364a800640 1 -- 172.21.15.187:0/1131764917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3644073900 msgr2=0x7f36440cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f364a800640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3644073900 0x7f36440cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f364a800640 1 -- 172.21.15.187:0/1131764917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364406be60 msgr2=0x7f36440b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f364a800640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364406be60 0x7f36440b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f364a800640 1 -- 172.21.15.187:0/1131764917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364414a680 con 0x7f364415ea60 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f3649400640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3644073900 0x7f36440cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f3649e00640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364406be60 0x7f36440b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f364a800640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364415ea60 0x7f364414a140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f364000c7b0 tx=0x7f364000cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.628+0000 7f363e000640 1 -- 172.21.15.187:0/1131764917 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640017070 con 0x7f364415ea60 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.629+0000 7f363e000640 1 -- 172.21.15.187:0/1131764917 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f364000a640 con 0x7f364415ea60 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.629+0000 7f363e000640 1 -- 172.21.15.187:0/1131764917 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364000a9f0 con 0x7f364415ea60 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.629+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36440cca50 con 0x7f364415ea60 2024-04-15T15:23:32.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.629+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36440cd010 con 0x7f364415ea60 2024-04-15T15:23:32.633 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.629+0000 7f363e000640 1 -- 172.21.15.187:0/1131764917 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f364001a070 con 0x7f364415ea60 2024-04-15T15:23:32.633 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.629+0000 7f363e000640 1 -- 172.21.15.187:0/1131764917 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f3640007840 con 0x7f364415ea60 2024-04-15T15:23:32.633 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.630+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36440755c0 con 0x7f364415ea60 2024-04-15T15:23:32.634 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.632+0000 7f363e000640 1 -- 172.21.15.187:0/1131764917 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f364414a680 con 0x7f364415ea60 2024-04-15T15:23:32.713 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.711+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fd9d396e-995d-4e3b-87b2-677311a2d3dc", "id": 3} v 0) -- 0x7f36440b6080 con 0x7f364415ea60 2024-04-15T15:23:32.721 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.719+0000 7f363e000640 1 -- 172.21.15.187:0/1131764917 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fd9d396e-995d-4e3b-87b2-677311a2d3dc", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f364000ab90 con 0x7f364415ea60 2024-04-15T15:23:32.721 INFO:teuthology.orchestra.run.smithi187.stdout:3 2024-04-15T15:23:32.722 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.720+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364415ea60 msgr2=0x7f364414a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.723 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.720+0000 7f364b200640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364415ea60 0x7f364414a140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f364000c7b0 tx=0x7f364000cc70 comp rx=0 tx=0).stop 2024-04-15T15:23:32.723 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.720+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 shutdown_connections 2024-04-15T15:23:32.723 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.720+0000 7f364b200640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364415ea60 0x7f364414a140 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.723 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.720+0000 7f364b200640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3644073900 0x7f36440cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.723 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.720+0000 7f364b200640 1 --2- 172.21.15.187:0/1131764917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364406be60 0x7f36440b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.723 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.720+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 >> 172.21.15.187:0/1131764917 conn(0x7f364406ca40 msgr2=0x7f36440b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:32.723 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.721+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 shutdown_connections 2024-04-15T15:23:32.723 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.721+0000 7f364b200640 1 -- 172.21.15.187:0/1131764917 wait complete. 2024-04-15T15:23:32.738 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 03203baa-ce38-45e6-b112-6b81c463ac07 4 2024-04-15T15:23:32.769 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:32.767+0000 7f70d56da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-15T15:23:32.770 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:32.768+0000 7fa6781b8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-15T15:23:32.904 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:32.902+0000 7fa6781b8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-15T15:23:32.914 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.912+0000 7f51f1e00640 1 Processor -- start 2024-04-15T15:23:32.914 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.912+0000 7f51f1e00640 1 -- start start 2024-04-15T15:23:32.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f1e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51ec15eb40 0x7f51ec162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f1e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f51ec0d1650 0x7f51ec0d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f1e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51ec163400 con 0x7f51ec0ba460 2024-04-15T15:23:32.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f1e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f51ec162e40 con 0x7f51ec0d1650 2024-04-15T15:23:32.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f1e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f51ec163120 con 0x7f51ec15eb40 2024-04-15T15:23:32.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f0a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51ec15eb40 0x7f51ec162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f0a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51ec15eb40 0x7f51ec162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46146/0 (socket says 172.21.15.187:46146) 2024-04-15T15:23:32.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f0a00640 1 -- 172.21.15.187:0/200160877 learned_addr learned my addr 172.21.15.187:0/200160877 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:32.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.914+0000 7f51f1400640 1 --2- 172.21.15.187:0/200160877 >> v2:172.21.15.150:3300/0 conn(0x7f51ec0d1650 0x7f51ec0d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.915+0000 7f51f0a00640 1 -- 172.21.15.187:0/200160877 >> v2:172.21.15.150:3300/0 conn(0x7f51ec0d1650 msgr2=0x7f51ec0d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:32.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.915+0000 7f51f0a00640 1 --2- 172.21.15.187:0/200160877 >> v2:172.21.15.150:3300/0 conn(0x7f51ec0d1650 0x7f51ec0d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.915+0000 7f51f0a00640 1 -- 172.21.15.187:0/200160877 >> v1:172.21.15.187:6789/0 conn(0x7f51ec0ba460 legacy=0x7f51ec0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.915+0000 7f51f0a00640 1 -- 172.21.15.187:0/200160877 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec0d2bc0 con 0x7f51ec15eb40 2024-04-15T15:23:32.918 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:32.915+0000 7f70d56da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-15T15:23:32.918 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.916+0000 7f51f0a00640 1 --2- 172.21.15.187:0/200160877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec15eb40 0x7f51ec162510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f51ec073890 tx=0x7f51e002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55e47f25c11e91b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:32.918 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.916+0000 7f51eb400640 1 -- 172.21.15.187:0/200160877 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0035020 con 0x7f51ec15eb40 2024-04-15T15:23:32.919 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.916+0000 7f51eb400640 1 -- 172.21.15.187:0/200160877 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e0031870 con 0x7f51ec15eb40 2024-04-15T15:23:32.919 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.916+0000 7f51eb400640 1 -- 172.21.15.187:0/200160877 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0031c40 con 0x7f51ec15eb40 2024-04-15T15:23:32.919 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.917+0000 7f51f1e00640 1 -- 172.21.15.187:0/200160877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec15eb40 msgr2=0x7f51ec162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.919 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.917+0000 7f51f1e00640 1 --2- 172.21.15.187:0/200160877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec15eb40 0x7f51ec162510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f51ec073890 tx=0x7f51e002b910 comp rx=0 tx=0).stop 2024-04-15T15:23:32.920 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.917+0000 7f51f1e00640 1 -- 172.21.15.187:0/200160877 shutdown_connections 2024-04-15T15:23:32.920 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.917+0000 7f51f1e00640 1 --2- 172.21.15.187:0/200160877 >> v2:172.21.15.150:3300/0 conn(0x7f51ec0d1650 0x7f51ec0d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.920 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.917+0000 7f51f1e00640 1 --2- 172.21.15.187:0/200160877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec15eb40 0x7f51ec162510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.920 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.917+0000 7f51f1e00640 1 -- 172.21.15.187:0/200160877 >> 172.21.15.187:0/200160877 conn(0x7f51ec06d3d0 msgr2=0x7f51ec073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:32.920 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.917+0000 7f51f1e00640 1 -- 172.21.15.187:0/200160877 shutdown_connections 2024-04-15T15:23:32.920 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.917+0000 7f51f1e00640 1 -- 172.21.15.187:0/200160877 wait complete. 2024-04-15T15:23:32.921 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.919+0000 7f51f1e00640 1 Processor -- start 2024-04-15T15:23:32.921 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.919+0000 7f51f1e00640 1 -- start start 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51f1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ec0ba460 0x7f51ec0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51f1e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec0d1650 0x7f51ec159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51f1e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51ec15eb40 0x7f51ec15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51f1e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec162a50 con 0x7f51ec0ba460 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51f1e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec0b7860 con 0x7f51ec0d1650 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51f1e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec162bd0 con 0x7f51ec15eb40 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec0d1650 0x7f51ec159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec0d1650 0x7f51ec159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46154/0 (socket says 172.21.15.187:46154) 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51ebe00640 1 -- 172.21.15.187:0/383108521 learned_addr learned my addr 172.21.15.187:0/383108521 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51f1400640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51ec15eb40 0x7f51ec15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.923 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.920+0000 7f51f0a00640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ec0ba460 0x7f51ec0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:32.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51ebe00640 1 -- 172.21.15.187:0/383108521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51ec15eb40 msgr2=0x7f51ec15ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51ebe00640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51ec15eb40 0x7f51ec15ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51ebe00640 1 -- 172.21.15.187:0/383108521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ec0ba460 msgr2=0x7f51ec0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:32.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51ebe00640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ec0ba460 0x7f51ec0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:32.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51ebe00640 1 -- 172.21.15.187:0/383108521 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec15f460 con 0x7f51ec0d1650 2024-04-15T15:23:32.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51ebe00640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec0d1650 0x7f51ec159d00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c440 tx=0x7f51d8001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:32.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51e8c00640 1 -- 172.21.15.187:0/383108521 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80052d0 con 0x7f51ec0d1650 2024-04-15T15:23:32.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51e8c00640 1 -- 172.21.15.187:0/383108521 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d8005470 con 0x7f51ec0d1650 2024-04-15T15:23:32.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51e8c00640 1 -- 172.21.15.187:0/383108521 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8007260 con 0x7f51ec0d1650 2024-04-15T15:23:32.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec15f700 con 0x7f51ec0d1650 2024-04-15T15:23:32.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.921+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec15fcc0 con 0x7f51ec0d1650 2024-04-15T15:23:32.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.922+0000 7f51e8c00640 1 -- 172.21.15.187:0/383108521 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f51d8007400 con 0x7f51ec0d1650 2024-04-15T15:23:32.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.922+0000 7f51e8c00640 1 -- 172.21.15.187:0/383108521 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f51d80035f0 con 0x7f51ec0d1650 2024-04-15T15:23:32.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.922+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec0755c0 con 0x7f51ec0d1650 2024-04-15T15:23:32.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:32.925+0000 7f51e8c00640 1 -- 172.21.15.187:0/383108521 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f51d8003aa0 con 0x7f51ec0d1650 2024-04-15T15:23:32.962 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:32.959+0000 7fa6781b8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-15T15:23:32.972 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:32.971+0000 7f70d56da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-15T15:23:33.005 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.003+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "03203baa-ce38-45e6-b112-6b81c463ac07", "id": 4} v 0) -- 0x7f51ec0d2360 con 0x7f51ec0d1650 2024-04-15T15:23:33.013 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.011+0000 7f51e8c00640 1 -- 172.21.15.187:0/383108521 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "03203baa-ce38-45e6-b112-6b81c463ac07", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f51d8003c80 con 0x7f51ec0d1650 2024-04-15T15:23:33.013 INFO:teuthology.orchestra.run.smithi187.stdout:4 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.012+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec0d1650 msgr2=0x7f51ec159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.012+0000 7f51f1e00640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec0d1650 0x7f51ec159d00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c440 tx=0x7f51d8001e70 comp rx=0 tx=0).stop 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.013+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 shutdown_connections 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.013+0000 7f51f1e00640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51ec15eb40 0x7f51ec15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.013+0000 7f51f1e00640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51ec0d1650 0x7f51ec159d00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.013+0000 7f51f1e00640 1 --2- 172.21.15.187:0/383108521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ec0ba460 0x7f51ec0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.013+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 >> 172.21.15.187:0/383108521 conn(0x7f51ec06d3d0 msgr2=0x7f51ec0b9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.013+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 shutdown_connections 2024-04-15T15:23:33.015 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.013+0000 7f51f1e00640 1 -- 172.21.15.187:0/383108521 wait complete. 2024-04-15T15:23:33.031 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 969f8b07-ebef-468f-b483-2423dedd8bdc 5 2024-04-15T15:23:33.117 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:33.100+0000 7f70d56da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:23:33.117 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:33.093+0000 7fa6781b8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:23:33.214 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:33.211+0000 7fa6781b8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-15T15:23:33.216 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:33.214+0000 7f70d56da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-15T15:23:33.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.309+0000 7f4f5a000640 1 Processor -- start 2024-04-15T15:23:33.311 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.309+0000 7f4f5a000640 1 -- start start 2024-04-15T15:23:33.313 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f5a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f5415e9a0 0x7f4f54162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.313 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f5a000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4f540cfc00 0x7f4f540d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.313 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f5a000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f54163450 con 0x7f4f540ba3e0 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f5a000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4f54162e90 con 0x7f4f540cfc00 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f5a000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4f54163170 con 0x7f4f5415e9a0 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f58c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f5415e9a0 0x7f4f54162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f58c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f5415e9a0 0x7f4f54162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58202/0 (socket says 172.21.15.187:58202) 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f58c00640 1 -- 172.21.15.187:0/3900480819 learned_addr learned my addr 172.21.15.187:0/3900480819 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f58c00640 1 -- 172.21.15.187:0/3900480819 >> v2:172.21.15.187:3300/0 conn(0x7f4f540cfc00 msgr2=0x7f4f540d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f58c00640 1 --2- 172.21.15.187:0/3900480819 >> v2:172.21.15.187:3300/0 conn(0x7f4f540cfc00 0x7f4f540d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f58c00640 1 -- 172.21.15.187:0/3900480819 >> v1:172.21.15.150:6789/0 conn(0x7f4f540ba3e0 legacy=0x7f4f540cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:33.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.311+0000 7f4f58c00640 1 -- 172.21.15.187:0/3900480819 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f540d2920 con 0x7f4f5415e9a0 2024-04-15T15:23:33.315 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.312+0000 7f4f58c00640 1 --2- 172.21.15.187:0/3900480819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f5415e9a0 0x7f4f54162590 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009870 tx=0x7f4f4802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e1eb27e8a380d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:33.315 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.313+0000 7f4f53400640 1 -- 172.21.15.187:0/3900480819 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48034070 con 0x7f4f5415e9a0 2024-04-15T15:23:33.315 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.313+0000 7f4f53400640 1 -- 172.21.15.187:0/3900480819 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f48030840 con 0x7f4f5415e9a0 2024-04-15T15:23:33.315 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.313+0000 7f4f53400640 1 -- 172.21.15.187:0/3900480819 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48030c10 con 0x7f4f5415e9a0 2024-04-15T15:23:33.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.313+0000 7f4f5a000640 1 -- 172.21.15.187:0/3900480819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f5415e9a0 msgr2=0x7f4f54162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.313+0000 7f4f5a000640 1 --2- 172.21.15.187:0/3900480819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f5415e9a0 0x7f4f54162590 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009870 tx=0x7f4f4802b980 comp rx=0 tx=0).stop 2024-04-15T15:23:33.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.314+0000 7f4f5a000640 1 -- 172.21.15.187:0/3900480819 shutdown_connections 2024-04-15T15:23:33.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.314+0000 7f4f5a000640 1 --2- 172.21.15.187:0/3900480819 >> v2:172.21.15.187:3300/0 conn(0x7f4f540cfc00 0x7f4f540d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.314+0000 7f4f5a000640 1 --2- 172.21.15.187:0/3900480819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f5415e9a0 0x7f4f54162590 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.314+0000 7f4f5a000640 1 -- 172.21.15.187:0/3900480819 >> 172.21.15.187:0/3900480819 conn(0x7f4f5406ce40 msgr2=0x7f4f5406d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:33.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.314+0000 7f4f5a000640 1 -- 172.21.15.187:0/3900480819 shutdown_connections 2024-04-15T15:23:33.316 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.314+0000 7f4f5a000640 1 -- 172.21.15.187:0/3900480819 wait complete. 2024-04-15T15:23:33.317 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.315+0000 7f4f5a000640 1 Processor -- start 2024-04-15T15:23:33.317 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.315+0000 7f4f5a000640 1 -- start start 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.316+0000 7f4f5a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f540ba3e0 0x7f4f54160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.316+0000 7f4f5a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f540cfc00 0x7f4f5414dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.316+0000 7f4f5a000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5415e9a0 0x7f4f541617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.316+0000 7f4f5a000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5415ed80 con 0x7f4f540ba3e0 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.316+0000 7f4f5a000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54162ad0 con 0x7f4f5415e9a0 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.316+0000 7f4f5a000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5415f570 con 0x7f4f540cfc00 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.316+0000 7f4f53e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f540cfc00 0x7f4f5414dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f53e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f540cfc00 0x7f4f5414dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58212/0 (socket says 172.21.15.187:58212) 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f53e00640 1 -- 172.21.15.187:0/1908636686 learned_addr learned my addr 172.21.15.187:0/1908636686 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:33.320 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f59600640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5415e9a0 0x7f4f541617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f58c00640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f540ba3e0 0x7f4f54160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f53e00640 1 -- 172.21.15.187:0/1908636686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5415e9a0 msgr2=0x7f4f541617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f53e00640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5415e9a0 0x7f4f541617f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f53e00640 1 -- 172.21.15.187:0/1908636686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f540ba3e0 msgr2=0x7f4f54160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f53e00640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f540ba3e0 0x7f4f54160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f53e00640 1 -- 172.21.15.187:0/1908636686 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5415f850 con 0x7f4f540cfc00 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f53e00640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f540cfc00 0x7f4f5414dba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c460 tx=0x7f4f4000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.317+0000 7f4f50c00640 1 -- 172.21.15.187:0/1908636686 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40016020 con 0x7f4f540cfc00 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.318+0000 7f4f50c00640 1 -- 172.21.15.187:0/1908636686 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f40012870 con 0x7f4f540cfc00 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.318+0000 7f4f50c00640 1 -- 172.21.15.187:0/1908636686 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40012c60 con 0x7f4f540cfc00 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.318+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5415fb30 con 0x7f4f540cfc00 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.318+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541600f0 con 0x7f4f540cfc00 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.318+0000 7f4f50c00640 1 -- 172.21.15.187:0/1908636686 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4f40019020 con 0x7f4f540cfc00 2024-04-15T15:23:33.321 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.318+0000 7f4f50c00640 1 -- 172.21.15.187:0/1908636686 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f4f40012e00 con 0x7f4f540cfc00 2024-04-15T15:23:33.322 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.319+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f540d0c80 con 0x7f4f540cfc00 2024-04-15T15:23:33.325 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.323+0000 7f4f50c00640 1 -- 172.21.15.187:0/1908636686 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4f540d0c80 con 0x7f4f540cfc00 2024-04-15T15:23:33.396 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:33.394+0000 7f70d56da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:23:33.400 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:33.398+0000 7fa6781b8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-15T15:23:33.404 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.402+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "969f8b07-ebef-468f-b483-2423dedd8bdc", "id": 5} v 0) -- 0x7f4f541602f0 con 0x7f4f540cfc00 2024-04-15T15:23:33.413 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.411+0000 7f4f50c00640 1 -- 172.21.15.187:0/1908636686 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "969f8b07-ebef-468f-b483-2423dedd8bdc", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f4f541602f0 con 0x7f4f540cfc00 2024-04-15T15:23:33.413 INFO:teuthology.orchestra.run.smithi187.stdout:5 2024-04-15T15:23:33.414 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f540cfc00 msgr2=0x7f4f5414dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f540cfc00 0x7f4f5414dba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c460 tx=0x7f4f4000c920 comp rx=0 tx=0).stop 2024-04-15T15:23:33.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 shutdown_connections 2024-04-15T15:23:33.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5415e9a0 0x7f4f541617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f540cfc00 0x7f4f5414dba0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 --2- 172.21.15.187:0/1908636686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f540ba3e0 0x7f4f54160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 >> 172.21.15.187:0/1908636686 conn(0x7f4f5406ce40 msgr2=0x7f4f540b7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:33.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 shutdown_connections 2024-04-15T15:23:33.415 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.412+0000 7f4f5a000640 1 -- 172.21.15.187:0/1908636686 wait complete. 2024-04-15T15:23:33.430 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new a5f188e2-5dea-4b4a-b561-f4686758e7d3 6 2024-04-15T15:23:33.447 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:33.446+0000 7f70d56da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-15T15:23:33.454 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:33.452+0000 7fa6781b8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-15T15:23:33.608 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.605+0000 7f4b5da00640 1 Processor -- start 2024-04-15T15:23:33.608 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.605+0000 7f4b5da00640 1 -- start start 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b5da00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4b5815eb40 0x7f4b58162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b5da00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4b580ba460 0x7f4b580d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b5da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580d1480 0x7f4b580b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b5da00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f4b58163400 con 0x7f4b5815eb40 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b5da00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4b58162e40 con 0x7f4b580ba460 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b5da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4b58163120 con 0x7f4b580d1480 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b5d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580d1480 0x7f4b580b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b4fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4b580ba460 0x7f4b580d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b4fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4b580ba460 0x7f4b580d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46176/0 (socket says 172.21.15.187:46176) 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b5d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580d1480 0x7f4b580b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:58226/0 (socket says 172.21.15.187:58226) 2024-04-15T15:23:33.610 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.607+0000 7f4b4fe00640 1 -- 172.21.15.187:0/3905866113 learned_addr learned my addr 172.21.15.187:0/3905866113 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:33.611 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.608+0000 7f4b57e00640 1 --2- 172.21.15.187:0/3905866113 >> v2:172.21.15.150:3300/0 conn(0x7f4b5815eb40 0x7f4b58162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.611 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.608+0000 7f4b4fe00640 1 -- 172.21.15.187:0/3905866113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580d1480 msgr2=0x7f4b580b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:33.611 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.608+0000 7f4b4fe00640 1 --2- 172.21.15.187:0/3905866113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580d1480 0x7f4b580b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.611 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.608+0000 7f4b4fe00640 1 -- 172.21.15.187:0/3905866113 >> v2:172.21.15.150:3300/0 conn(0x7f4b5815eb40 msgr2=0x7f4b58162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:33.611 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.608+0000 7f4b4fe00640 1 --2- 172.21.15.187:0/3905866113 >> v2:172.21.15.150:3300/0 conn(0x7f4b5815eb40 0x7f4b58162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.611 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.608+0000 7f4b4fe00640 1 -- 172.21.15.187:0/3905866113 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b580d2bc0 con 0x7f4b580ba460 2024-04-15T15:23:33.611 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.608+0000 7f4b57e00640 1 --2- 172.21.15.187:0/3905866113 >> v2:172.21.15.150:3300/0 conn(0x7f4b5815eb40 0x7f4b58162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:23:33.612 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.609+0000 7f4b4fe00640 1 --2- 172.21.15.187:0/3905866113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580ba460 0x7f4b580d0f40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4b580ceca0 tx=0x7f4b4002b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940031de6195a92c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:33.612 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.609+0000 7f4b57400640 1 -- 172.21.15.187:0/3905866113 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4002e070 con 0x7f4b580ba460 2024-04-15T15:23:33.612 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.609+0000 7f4b57400640 1 -- 172.21.15.187:0/3905866113 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b40037070 con 0x7f4b580ba460 2024-04-15T15:23:33.612 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.609+0000 7f4b57400640 1 -- 172.21.15.187:0/3905866113 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40031a00 con 0x7f4b580ba460 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 -- 172.21.15.187:0/3905866113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580ba460 msgr2=0x7f4b580d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 --2- 172.21.15.187:0/3905866113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580ba460 0x7f4b580d0f40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4b580ceca0 tx=0x7f4b4002b880 comp rx=0 tx=0).stop 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 -- 172.21.15.187:0/3905866113 shutdown_connections 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 --2- 172.21.15.187:0/3905866113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580d1480 0x7f4b580b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 --2- 172.21.15.187:0/3905866113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580ba460 0x7f4b580d0f40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 --2- 172.21.15.187:0/3905866113 >> v2:172.21.15.150:3300/0 conn(0x7f4b5815eb40 0x7f4b58162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 -- 172.21.15.187:0/3905866113 >> 172.21.15.187:0/3905866113 conn(0x7f4b5806d3d0 msgr2=0x7f4b58073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 -- 172.21.15.187:0/3905866113 shutdown_connections 2024-04-15T15:23:33.613 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.610+0000 7f4b5da00640 1 -- 172.21.15.187:0/3905866113 wait complete. 2024-04-15T15:23:33.614 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.612+0000 7f4b5da00640 1 Processor -- start 2024-04-15T15:23:33.614 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.612+0000 7f4b5da00640 1 -- start start 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b5da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580ba460 0x7f4b58154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b5da00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580d1480 0x7f4b580d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b5da00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5815eb40 0x7f4b580d4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b5da00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4b58162a50 con 0x7f4b5815eb40 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b5da00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580b7860 con 0x7f4b580d1480 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b5da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4b58162bd0 con 0x7f4b580ba460 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b57e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580ba460 0x7f4b58154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580d1480 0x7f4b580d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.616 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580d1480 0x7f4b580d3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46178/0 (socket says 172.21.15.187:46178) 2024-04-15T15:23:33.617 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.613+0000 7f4b4fe00640 1 -- 172.21.15.187:0/3722252760 learned_addr learned my addr 172.21.15.187:0/3722252760 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:33.617 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b4fe00640 1 -- 172.21.15.187:0/3722252760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580ba460 msgr2=0x7f4b58154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.617 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b4fe00640 1 --2- 172.21.15.187:0/3722252760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580ba460 0x7f4b58154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.617 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b4fe00640 1 -- 172.21.15.187:0/3722252760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5815eb40 msgr2=0x7f4b580d4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.617 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b4fe00640 1 --2- 172.21.15.187:0/3722252760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5815eb40 0x7f4b580d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.617 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b4fe00640 1 -- 172.21.15.187:0/3722252760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b580cc210 con 0x7f4b580d1480 2024-04-15T15:23:33.617 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b4fe00640 1 --2- 172.21.15.187:0/3722252760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580d1480 0x7f4b580d3c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b40000c00 tx=0x7f4b400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:33.618 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b54c00640 1 -- 172.21.15.187:0/3722252760 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40032670 con 0x7f4b580d1480 2024-04-15T15:23:33.618 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b54c00640 1 -- 172.21.15.187:0/3722252760 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b40032850 con 0x7f4b580d1480 2024-04-15T15:23:33.618 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b580cca90 con 0x7f4b580d1480 2024-04-15T15:23:33.618 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.614+0000 7f4b54c00640 1 -- 172.21.15.187:0/3722252760 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40032c40 con 0x7f4b580d1480 2024-04-15T15:23:33.618 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.615+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b580cc7b0 con 0x7f4b580d1480 2024-04-15T15:23:33.618 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.615+0000 7f4b54c00640 1 -- 172.21.15.187:0/3722252760 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b4004c060 con 0x7f4b580d1480 2024-04-15T15:23:33.618 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.615+0000 7f4b54c00640 1 -- 172.21.15.187:0/3722252760 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f4b400459e0 con 0x7f4b580d1480 2024-04-15T15:23:33.619 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.616+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b580755c0 con 0x7f4b580d1480 2024-04-15T15:23:33.620 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.618+0000 7f4b54c00640 1 -- 172.21.15.187:0/3722252760 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4b40040070 con 0x7f4b580d1480 2024-04-15T15:23:33.625 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:33.623+0000 7f70d56da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-15T15:23:33.639 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:33.637+0000 7fa6781b8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-15T15:23:33.699 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.697+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a5f188e2-5dea-4b4a-b561-f4686758e7d3", "id": 6} v 0) -- 0x7f4b580d2360 con 0x7f4b580d1480 2024-04-15T15:23:33.706 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.704+0000 7f4b54c00640 1 -- 172.21.15.187:0/3722252760 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a5f188e2-5dea-4b4a-b561-f4686758e7d3", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f4b400453f0 con 0x7f4b580d1480 2024-04-15T15:23:33.707 INFO:teuthology.orchestra.run.smithi187.stdout:6 2024-04-15T15:23:33.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580d1480 msgr2=0x7f4b580d3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 --2- 172.21.15.187:0/3722252760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580d1480 0x7f4b580d3c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b40000c00 tx=0x7f4b400046c0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 shutdown_connections 2024-04-15T15:23:33.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 --2- 172.21.15.187:0/3722252760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5815eb40 0x7f4b580d4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 --2- 172.21.15.187:0/3722252760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b580d1480 0x7f4b580d3c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 --2- 172.21.15.187:0/3722252760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b580ba460 0x7f4b58154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 >> 172.21.15.187:0/3722252760 conn(0x7f4b5806d3d0 msgr2=0x7f4b580b9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:33.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 shutdown_connections 2024-04-15T15:23:33.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.706+0000 7f4b5da00640 1 -- 172.21.15.187:0/3722252760 wait complete. 2024-04-15T15:23:33.724 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 89666b85-6bae-44f7-b9b1-4a69c9f8e436 7 2024-04-15T15:23:33.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.952+0000 7f9f1dc00640 1 Processor -- start 2024-04-15T15:23:33.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.952+0000 7f9f1dc00640 1 -- start start 2024-04-15T15:23:33.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.953+0000 7f9f1dc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f180cfc00 0x7f9f181698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.953+0000 7f9f1dc00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f18163450 con 0x7f9f1815e9a0 2024-04-15T15:23:33.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.953+0000 7f9f1dc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f18162e90 con 0x7f9f180ba3e0 2024-04-15T15:23:33.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.953+0000 7f9f1dc00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f9f18163170 con 0x7f9f180cfc00 2024-04-15T15:23:33.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.953+0000 7f9f0fe00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f9f180ba3e0 0x7f9f180cf4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:48838/0 (socket says 172.21.15.187:48838) 2024-04-15T15:23:33.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.953+0000 7f9f0fe00640 1 -- 172.21.15.187:0/2930921969 learned_addr learned my addr 172.21.15.187:0/2930921969 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f1d200640 1 --2- 172.21.15.187:0/2930921969 >> v2:172.21.15.150:3300/0 conn(0x7f9f180cfc00 0x7f9f181698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 869775552 0 0) 0x7f9f18162e90 con 0x7f9f180ba3e0 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9efc003520 con 0x7f9f180ba3e0 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1645209367 0 0) 0x7f9f18163450 con 0x7f9f1815e9a0 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f18162e90 con 0x7f9f1815e9a0 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2844925392 0 0) 0x7f9efc003520 con 0x7f9f180ba3e0 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 >> v2:172.21.15.150:3300/0 conn(0x7f9f180cfc00 msgr2=0x7f9f181698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 --2- 172.21.15.187:0/2930921969 >> v2:172.21.15.150:3300/0 conn(0x7f9f180cfc00 0x7f9f181698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 >> v1:172.21.15.150:6789/0 conn(0x7f9f1815e9a0 legacy=0x7f9f18162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.958 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f180d2920 con 0x7f9f180ba3e0 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.954+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f9f00001670 con 0x7f9f180ba3e0 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 >> v1:172.21.15.187:6789/0 conn(0x7f9f180ba3e0 legacy=0x7f9f180cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17400640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc003b90 0x7f9efc003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17400640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17400640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9efc005210 0x7f9efc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18163170 con 0x7f9efc003b90 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc00c090 con 0x7f9efc004800 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9efc00c290 con 0x7f9efc005210 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f0fe00640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f1d200640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9efc005210 0x7f9efc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17e00640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc003b90 0x7f9efc003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f0fe00640 1 -- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9efc005210 msgr2=0x7f9efc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f0fe00640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9efc005210 0x7f9efc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f0fe00640 1 -- 172.21.15.187:0/2930921969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc003b90 msgr2=0x7f9efc003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:33.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f0fe00640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc003b90 0x7f9efc003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f0fe00640 1 -- 172.21.15.187:0/2930921969 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc003520 con 0x7f9efc004800 2024-04-15T15:23:33.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f1d200640 1 -- 172.21.15.187:0/2930921969 reap_dead start 2024-04-15T15:23:33.960 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.955+0000 7f9f17e00640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc003b90 0x7f9efc003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.956+0000 7f9f0fe00640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f00007fc0 tx=0x7f9efc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1949eab7227a1b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00007be0 con 0x7f9efc004800 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f00030070 con 0x7f9efc004800 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f17400640 1 -- 172.21.15.187:0/2930921969 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00005d30 con 0x7f9efc004800 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f1dc00640 1 -- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 msgr2=0x7f9efc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f1dc00640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f00007fc0 tx=0x7f9efc005830 comp rx=0 tx=0).stop 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f1dc00640 1 -- 172.21.15.187:0/2930921969 shutdown_connections 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f1dc00640 1 --2- 172.21.15.187:0/2930921969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f1dc00640 1 -- 172.21.15.187:0/2930921969 >> 172.21.15.187:0/2930921969 conn(0x7f9f1806ce40 msgr2=0x7f9f1806d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f1dc00640 1 -- 172.21.15.187:0/2930921969 shutdown_connections 2024-04-15T15:23:33.961 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.957+0000 7f9f1dc00640 1 -- 172.21.15.187:0/2930921969 wait complete. 2024-04-15T15:23:33.962 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.960+0000 7f9f1dc00640 1 Processor -- start 2024-04-15T15:23:33.962 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.960+0000 7f9f1dc00640 1 -- start start 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f1dc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9f18149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f1dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f18149c50 0x7f9f18161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f1dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180d1460 0x7f9f180d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f1dc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1815ed80 con 0x7f9f18149c50 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f1dc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f180cf4f0 con 0x7f9efc004800 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f1dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f180c8f50 con 0x7f9f180d1460 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9f18149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9f18149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:41310/0 (socket says 172.21.15.187:41310) 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f17e00640 1 -- 172.21.15.187:0/3013107777 learned_addr learned my addr 172.21.15.187:0/3013107777 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f1d200640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180d1460 0x7f9f180d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.962+0000 7f9f0fe00640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f18149c50 0x7f9f18161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:33.966 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f17e00640 1 -- 172.21.15.187:0/3013107777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180d1460 msgr2=0x7f9f180d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f17e00640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180d1460 0x7f9f180d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f17e00640 1 -- 172.21.15.187:0/3013107777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f18149c50 msgr2=0x7f9f18161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f17e00640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f18149c50 0x7f9f18161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f17e00640 1 -- 172.21.15.187:0/3013107777 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f18149990 con 0x7f9efc004800 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f17e00640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9f18149450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e580 tx=0x7f9f0800ea40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f14c00640 1 -- 172.21.15.187:0/3013107777 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f080052e0 con 0x7f9efc004800 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f14c00640 1 -- 172.21.15.187:0/3013107777 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f080067d0 con 0x7f9efc004800 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f14c00640 1 -- 172.21.15.187:0/3013107777 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08006b80 con 0x7f9efc004800 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.963+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f180c8340 con 0x7f9efc004800 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.964+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f180c8900 con 0x7f9efc004800 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.964+0000 7f9f14c00640 1 -- 172.21.15.187:0/3013107777 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f08007030 con 0x7f9efc004800 2024-04-15T15:23:33.967 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.964+0000 7f9f14c00640 1 -- 172.21.15.187:0/3013107777 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f9f08006d20 con 0x7f9efc004800 2024-04-15T15:23:33.968 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.965+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f18155230 con 0x7f9efc004800 2024-04-15T15:23:33.969 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:33.967+0000 7f9f14c00640 1 -- 172.21.15.187:0/3013107777 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9f18155230 con 0x7f9efc004800 2024-04-15T15:23:34.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.045+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "89666b85-6bae-44f7-b9b1-4a69c9f8e436", "id": 7} v 0) -- 0x7f9f180d0100 con 0x7f9efc004800 2024-04-15T15:23:34.058 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.056+0000 7f9f14c00640 1 -- 172.21.15.187:0/3013107777 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89666b85-6bae-44f7-b9b1-4a69c9f8e436", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f9f08021020 con 0x7f9efc004800 2024-04-15T15:23:34.058 INFO:teuthology.orchestra.run.smithi187.stdout:7 2024-04-15T15:23:34.060 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.057+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 msgr2=0x7f9f18149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:34.060 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.057+0000 7f9f1dc00640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9f18149450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e580 tx=0x7f9f0800ea40 comp rx=0 tx=0).stop 2024-04-15T15:23:34.060 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.058+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 shutdown_connections 2024-04-15T15:23:34.060 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.058+0000 7f9f1dc00640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f180d1460 0x7f9f180d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.060 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.058+0000 7f9f1dc00640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f18149c50 0x7f9f18161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.060 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.058+0000 7f9f1dc00640 1 --2- 172.21.15.187:0/3013107777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc004800 0x7f9f18149450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.060 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.058+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 >> 172.21.15.187:0/3013107777 conn(0x7f9f1806ce40 msgr2=0x7f9f180cb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:34.061 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.058+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 shutdown_connections 2024-04-15T15:23:34.061 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:23:34.058+0000 7f9f1dc00640 1 -- 172.21.15.187:0/3013107777 wait complete. 2024-04-15T15:23:34.077 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-15T15:23:34.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-15T15:23:34.085 INFO:tasks.ceph.osd.0:Started 2024-04-15T15:23:34.085 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-15T15:23:34.085 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-15T15:23:34.092 INFO:tasks.ceph.osd.1:Started 2024-04-15T15:23:34.092 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-15T15:23:34.092 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-15T15:23:34.099 INFO:tasks.ceph.osd.2:Started 2024-04-15T15:23:34.099 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-15T15:23:34.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-15T15:23:34.105 INFO:tasks.ceph.osd.3:Started 2024-04-15T15:23:34.105 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-15T15:23:34.105 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-15T15:23:34.110 INFO:tasks.ceph.osd.4:Started 2024-04-15T15:23:34.110 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-15T15:23:34.110 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-15T15:23:34.114 INFO:tasks.ceph.osd.5:Started 2024-04-15T15:23:34.114 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-15T15:23:34.115 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-15T15:23:34.119 INFO:tasks.ceph.osd.6:Started 2024-04-15T15:23:34.119 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-15T15:23:34.119 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-15T15:23:34.124 INFO:tasks.ceph.osd.7:Started 2024-04-15T15:23:34.124 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:34.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.334+0000 7fc7efe00640 1 Processor -- start 2024-04-15T15:23:34.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.335+0000 7fc7efe00640 1 -- start start 2024-04-15T15:23:34.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7efe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc7e81461f0 0x7fc7e80ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:34.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e80b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:34.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7efe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc7e80b3e10 0x7fc7e80cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:34.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7efe00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e80a9640 con 0x7fc7e80b3e10 2024-04-15T15:23:34.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7efe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e80cf250 con 0x7fc7e81461f0 2024-04-15T15:23:34.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7efe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e80b4be0 con 0x7fc7e80cedf0 2024-04-15T15:23:34.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7ede00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc7e80b3e10 0x7fc7e80cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:34.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7ede00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc7e80b3e10 0x7fc7e80cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43302/0 (socket says 172.21.15.150:43302) 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7ede00640 1 -- 172.21.15.150:0/323509246 learned_addr learned my addr 172.21.15.150:0/323509246 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7ed400640 1 --2- 172.21.15.150:0/323509246 >> v2:172.21.15.187:3300/0 conn(0x7fc7e81461f0 0x7fc7e80ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.336+0000 7fc7eca00640 1 --2- 172.21.15.150:0/323509246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e80b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.337+0000 7fc7ede00640 1 -- 172.21.15.150:0/323509246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 msgr2=0x7fc7e80b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.337+0000 7fc7ede00640 1 --2- 172.21.15.150:0/323509246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e80b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.337+0000 7fc7ede00640 1 -- 172.21.15.150:0/323509246 >> v2:172.21.15.187:3300/0 conn(0x7fc7e81461f0 msgr2=0x7fc7e80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.337+0000 7fc7ede00640 1 --2- 172.21.15.150:0/323509246 >> v2:172.21.15.187:3300/0 conn(0x7fc7e81461f0 0x7fc7e80ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.337+0000 7fc7ede00640 1 -- 172.21.15.150:0/323509246 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e80b4620 con 0x7fc7e80b3e10 2024-04-15T15:23:34.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.338+0000 7fc7ede00640 1 --2- 172.21.15.150:0/323509246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e80b3e10 0x7fc7e80cd2b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009e50 tx=0x7fc7e002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44628b6631d2ca3a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:34.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.338+0000 7fc7e7e00640 1 -- 172.21.15.150:0/323509246 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e002ca70 con 0x7fc7e80b3e10 2024-04-15T15:23:34.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.338+0000 7fc7e7e00640 1 -- 172.21.15.150:0/323509246 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e002cc10 con 0x7fc7e80b3e10 2024-04-15T15:23:34.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.338+0000 7fc7e7e00640 1 -- 172.21.15.150:0/323509246 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00343f0 con 0x7fc7e80b3e10 2024-04-15T15:23:34.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.339+0000 7fc7efe00640 1 -- 172.21.15.150:0/323509246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e80b3e10 msgr2=0x7fc7e80cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:34.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.339+0000 7fc7efe00640 1 --2- 172.21.15.150:0/323509246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e80b3e10 0x7fc7e80cd2b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009e50 tx=0x7fc7e002bed0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.339+0000 7fc7efe00640 1 -- 172.21.15.150:0/323509246 shutdown_connections 2024-04-15T15:23:34.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.339+0000 7fc7efe00640 1 --2- 172.21.15.150:0/323509246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e80b3e10 0x7fc7e80cd2b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.339+0000 7fc7efe00640 1 --2- 172.21.15.150:0/323509246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e80b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.339+0000 7fc7efe00640 1 --2- 172.21.15.150:0/323509246 >> v2:172.21.15.187:3300/0 conn(0x7fc7e81461f0 0x7fc7e80ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.339+0000 7fc7efe00640 1 -- 172.21.15.150:0/323509246 >> 172.21.15.150:0/323509246 conn(0x7fc7e80b27b0 msgr2=0x7fc7e80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:34.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.340+0000 7fc7efe00640 1 -- 172.21.15.150:0/323509246 shutdown_connections 2024-04-15T15:23:34.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.340+0000 7fc7efe00640 1 -- 172.21.15.150:0/323509246 wait complete. 2024-04-15T15:23:34.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.342+0000 7fc7efe00640 1 Processor -- start 2024-04-15T15:23:34.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.342+0000 7fc7efe00640 1 -- start start 2024-04-15T15:23:34.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.343+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e80b3e10 0x7fc7e8140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:34.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.343+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e813ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:34.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e81461f0 0x7fc7e8158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:34.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7efe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e815d290 con 0x7fc7e81461f0 2024-04-15T15:23:34.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7efe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e815cfb0 con 0x7fc7e80b3e10 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7efe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e81465d0 con 0x7fc7e80cedf0 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ede00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e81461f0 0x7fc7e8158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ede00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e81461f0 0x7fc7e8158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43304/0 (socket says 172.21.15.150:43304) 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ede00640 1 -- 172.21.15.150:0/550731556 learned_addr learned my addr 172.21.15.150:0/550731556 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7eca00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e813ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ed400640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e80b3e10 0x7fc7e8140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ede00640 1 -- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 msgr2=0x7fc7e813ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ede00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e813ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ede00640 1 -- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e80b3e10 msgr2=0x7fc7e8140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ede00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e80b3e10 0x7fc7e8140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7ede00640 1 -- 172.21.15.150:0/550731556 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e813b4e0 con 0x7fc7e81461f0 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.344+0000 7fc7eca00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e813ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.345+0000 7fc7ede00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e81461f0 0x7fc7e8158240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009e50 tx=0x7fc7e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:34.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.345+0000 7fc7e5600640 1 -- 172.21.15.150:0/550731556 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0043020 con 0x7fc7e81461f0 2024-04-15T15:23:34.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.345+0000 7fc7e5600640 1 -- 172.21.15.150:0/550731556 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e003e450 con 0x7fc7e81461f0 2024-04-15T15:23:34.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.345+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e815a240 con 0x7fc7e81461f0 2024-04-15T15:23:34.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.345+0000 7fc7e5600640 1 -- 172.21.15.150:0/550731556 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0009510 con 0x7fc7e81461f0 2024-04-15T15:23:34.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.345+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e815a800 con 0x7fc7e81461f0 2024-04-15T15:23:34.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.346+0000 7fc7e5600640 1 -- 172.21.15.150:0/550731556 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc7e0005d10 con 0x7fc7e81461f0 2024-04-15T15:23:34.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.346+0000 7fc7e5600640 1 -- 172.21.15.150:0/550731556 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc7e0048ae0 con 0x7fc7e81461f0 2024-04-15T15:23:34.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.346+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e80704e0 con 0x7fc7e81461f0 2024-04-15T15:23:34.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.350+0000 7fc7e5600640 1 -- 172.21.15.150:0/550731556 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc7e80704e0 con 0x7fc7e81461f0 2024-04-15T15:23:34.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.415+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7e8068c80 con 0x7fc7e81461f0 2024-04-15T15:23:34.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.416+0000 7fc7e5600640 1 -- 172.21.15.150:0/550731556 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc7e0048d30 con 0x7fc7e81461f0 2024-04-15T15:23:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:34.418 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":10,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:34.049102+0000","last_up_change":"0.000000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e81461f0 msgr2=0x7fc7e8158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e81461f0 0x7fc7e8158240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009e50 tx=0x7fc7e0004a40 comp rx=0 tx=0).stop 2024-04-15T15:23:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 shutdown_connections 2024-04-15T15:23:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e81461f0 0x7fc7e8158240 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e80cedf0 0x7fc7e813ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 --2- 172.21.15.150:0/550731556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e80b3e10 0x7fc7e8140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 >> 172.21.15.150:0/550731556 conn(0x7fc7e80b27b0 msgr2=0x7fc7e815d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 shutdown_connections 2024-04-15T15:23:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:34.418+0000 7fc7efe00640 1 -- 172.21.15.150:0/550731556 wait complete. 2024-04-15T15:23:34.435 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-15T15:23:34.435 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-15T15:23:34.676 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:34.674+0000 7fa6781b8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-15T15:23:34.698 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:34.696+0000 7f70d56da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-15T15:23:34.787 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-04-15T15:23:34.785+0000 7fe2c3ef9640 -1 Falling back to public interface 2024-04-15T15:23:34.792 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-04-15T15:23:34.790+0000 7f22e04d9640 -1 Falling back to public interface 2024-04-15T15:23:34.797 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-15T15:23:34.795+0000 7fd845949640 -1 Falling back to public interface 2024-04-15T15:23:34.802 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-15T15:23:34.799+0000 7f4432ef9640 -1 Falling back to public interface 2024-04-15T15:23:34.802 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-15T15:23:34.800+0000 7fe29fb1d640 -1 Falling back to public interface 2024-04-15T15:23:34.802 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-15T15:23:34.800+0000 7f4b9400e640 -1 Falling back to public interface 2024-04-15T15:23:34.803 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-15T15:23:34.801+0000 7f4b92400640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6810/0: (98) Address already in use 2024-04-15T15:23:34.810 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-04-15T15:23:34.809+0000 7fcf51e06640 -1 Falling back to public interface 2024-04-15T15:23:34.817 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-04-15T15:23:34.816+0000 7f8ae5410640 -1 Falling back to public interface 2024-04-15T15:23:34.993 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:34.991+0000 7fa6781b8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-15T15:23:35.026 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:35.024+0000 7f70d56da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-15T15:23:35.053 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:35.051+0000 7fa6781b8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-15T15:23:35.088 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:35.086+0000 7f70d56da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-15T15:23:35.189 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:35.187+0000 7fa6781b8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-15T15:23:35.225 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-04-15T15:23:35.223+0000 7f70d56da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-15T15:23:35.241 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:35.389 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.387+0000 7f418da00640 1 Processor -- start 2024-04-15T15:23:35.389 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.387+0000 7f418da00640 1 -- start start 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.388+0000 7f418da00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4188146370 0x7f418806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.388+0000 7f418da00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f418806e790 0x7f41880721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.388+0000 7f418da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4188072730 0x7f41880cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.388+0000 7f418da00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f41880a9630 con 0x7f4188146370 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.388+0000 7f418da00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4188072b60 con 0x7f418806e790 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.388+0000 7f418da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f418815ce50 con 0x7f4188072730 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4188146370 0x7f418806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4188146370 0x7f418806e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43306/0 (socket says 172.21.15.150:43306) 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186a00640 1 -- 172.21.15.150:0/1757164018 learned_addr learned my addr 172.21.15.150:0/1757164018 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4187400640 1 --2- 172.21.15.150:0/1757164018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4188072730 0x7f41880cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186000640 1 --2- 172.21.15.150:0/1757164018 >> v2:172.21.15.187:3300/0 conn(0x7f418806e790 0x7f41880721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:35.391 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186a00640 1 -- 172.21.15.150:0/1757164018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4188072730 msgr2=0x7f41880cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:35.392 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186a00640 1 --2- 172.21.15.150:0/1757164018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4188072730 0x7f41880cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.392 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186a00640 1 -- 172.21.15.150:0/1757164018 >> v2:172.21.15.187:3300/0 conn(0x7f418806e790 msgr2=0x7f41880721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:35.392 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186a00640 1 --2- 172.21.15.150:0/1757164018 >> v2:172.21.15.187:3300/0 conn(0x7f418806e790 0x7f41880721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.392 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.389+0000 7f4186a00640 1 -- 172.21.15.150:0/1757164018 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418806eca0 con 0x7f4188146370 2024-04-15T15:23:35.393 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.390+0000 7f4186a00640 1 --2- 172.21.15.150:0/1757164018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188146370 0x7f418806e250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f418806fbb0 tx=0x7f417802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2765d15c7d9185f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:35.393 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.390+0000 7f4185600640 1 -- 172.21.15.150:0/1757164018 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178035020 con 0x7f4188146370 2024-04-15T15:23:35.393 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.390+0000 7f4185600640 1 -- 172.21.15.150:0/1757164018 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4178031640 con 0x7f4188146370 2024-04-15T15:23:35.393 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.390+0000 7f4185600640 1 -- 172.21.15.150:0/1757164018 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417802d040 con 0x7f4188146370 2024-04-15T15:23:35.393 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 -- 172.21.15.150:0/1757164018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188146370 msgr2=0x7f418806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:35.393 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 --2- 172.21.15.150:0/1757164018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188146370 0x7f418806e250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f418806fbb0 tx=0x7f417802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.393 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 -- 172.21.15.150:0/1757164018 shutdown_connections 2024-04-15T15:23:35.394 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 --2- 172.21.15.150:0/1757164018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4188072730 0x7f41880cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.394 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 --2- 172.21.15.150:0/1757164018 >> v2:172.21.15.187:3300/0 conn(0x7f418806e790 0x7f41880721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.394 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 --2- 172.21.15.150:0/1757164018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188146370 0x7f418806e250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.394 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 -- 172.21.15.150:0/1757164018 >> 172.21.15.150:0/1757164018 conn(0x7f41880b25c0 msgr2=0x7f41880b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:35.394 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 -- 172.21.15.150:0/1757164018 shutdown_connections 2024-04-15T15:23:35.394 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.391+0000 7f418da00640 1 -- 172.21.15.150:0/1757164018 wait complete. 2024-04-15T15:23:35.394 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.393+0000 7f418da00640 1 Processor -- start 2024-04-15T15:23:35.394 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.393+0000 7f418da00640 1 -- start start 2024-04-15T15:23:35.396 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f418da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f418806e790 0x7f418814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f418da00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188072730 0x7f41881f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f418da00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4188146370 0x7f4188074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f418da00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f418806f6d0 con 0x7f4188072730 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f418da00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f418815d0d0 con 0x7f4188146370 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f418da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f418806f850 con 0x7f418806e790 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188072730 0x7f41881f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188072730 0x7f41881f45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43312/0 (socket says 172.21.15.150:43312) 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186000640 1 -- 172.21.15.150:0/1414656195 learned_addr learned my addr 172.21.15.150:0/1414656195 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186a00640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f418806e790 0x7f418814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4187400640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4188146370 0x7f4188074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186000640 1 -- 172.21.15.150:0/1414656195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f418806e790 msgr2=0x7f418814fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186000640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f418806e790 0x7f418814fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.397 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186000640 1 -- 172.21.15.150:0/1414656195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4188146370 msgr2=0x7f4188074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:35.398 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186000640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4188146370 0x7f4188074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.398 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.394+0000 7f4186000640 1 -- 172.21.15.150:0/1414656195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41881f50d0 con 0x7f4188072730 2024-04-15T15:23:35.398 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.395+0000 7f4186000640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188072730 0x7f41881f45f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f417400d800 tx=0x7f417400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:35.398 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.395+0000 7f417ea00640 1 -- 172.21.15.150:0/1414656195 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174018020 con 0x7f4188072730 2024-04-15T15:23:35.398 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.395+0000 7f417ea00640 1 -- 172.21.15.150:0/1414656195 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41740096a0 con 0x7f4188072730 2024-04-15T15:23:35.398 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.395+0000 7f417ea00640 1 -- 172.21.15.150:0/1414656195 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174009a70 con 0x7f4188072730 2024-04-15T15:23:35.398 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.395+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41881f53b0 con 0x7f4188072730 2024-04-15T15:23:35.398 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.395+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41881f5970 con 0x7f4188072730 2024-04-15T15:23:35.399 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.396+0000 7f417ea00640 1 -- 172.21.15.150:0/1414656195 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 2) ==== 45545+0+0 (secure 0 0 0) 0x7f417401b020 con 0x7f4188072730 2024-04-15T15:23:35.399 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.396+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4188068f30 con 0x7f4188072730 2024-04-15T15:23:35.399 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.396+0000 7f417ea00640 1 -- 172.21.15.150:0/1414656195 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f417404b0a0 con 0x7f4188072730 2024-04-15T15:23:35.400 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.399+0000 7f417ea00640 1 -- 172.21.15.150:0/1414656195 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4174022020 con 0x7f4188072730 2024-04-15T15:23:35.464 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.462+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4188073c30 con 0x7f4188072730 2024-04-15T15:23:35.465 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.463+0000 7f417ea00640 1 -- 172.21.15.150:0/1414656195 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4174014930 con 0x7f4188072730 2024-04-15T15:23:35.465 INFO:teuthology.misc.health.smithi150.stdout: 2024-04-15T15:23:35.465 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":10,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:34.049102+0000","last_up_change":"0.000000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:35.466 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188072730 msgr2=0x7f41881f45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:35.466 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188072730 0x7f41881f45f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f417400d800 tx=0x7f417400dcc0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.467 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 shutdown_connections 2024-04-15T15:23:35.467 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4188146370 0x7f4188074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.467 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4188072730 0x7f41881f45f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.467 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 --2- 172.21.15.150:0/1414656195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f418806e790 0x7f418814fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:35.467 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 >> 172.21.15.150:0/1414656195 conn(0x7f41880b25c0 msgr2=0x7f41880b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:35.467 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 shutdown_connections 2024-04-15T15:23:35.467 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:35.465+0000 7f418da00640 1 -- 172.21.15.150:0/1414656195 wait complete. 2024-04-15T15:23:35.479 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-15T15:23:36.220 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:36.218+0000 7fa619200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-15T15:23:36.225 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:23:36.223+0000 7fa619200640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-15T15:23:38.133 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-04-15T15:23:38.131+0000 7f22e04d9640 -1 osd.2 0 log_to_monitors true 2024-04-15T15:23:38.391 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-15T15:23:38.389+0000 7fd845949640 -1 osd.5 0 log_to_monitors true 2024-04-15T15:23:38.400 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-04-15T15:23:38.399+0000 7fe2c3ef9640 -1 osd.1 0 log_to_monitors true 2024-04-15T15:23:38.418 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-04-15T15:23:38.416+0000 7fcf51e06640 -1 osd.3 0 log_to_monitors true 2024-04-15T15:23:38.427 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-04-15T15:23:38.426+0000 7f8ae5410640 -1 osd.0 0 log_to_monitors true 2024-04-15T15:23:38.460 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-15T15:23:38.458+0000 7fe29fb1d640 -1 osd.4 0 log_to_monitors true 2024-04-15T15:23:38.470 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-15T15:23:38.468+0000 7f4432ef9640 -1 osd.7 0 log_to_monitors true 2024-04-15T15:23:38.480 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-15T15:23:38.478+0000 7f4b9400e640 -1 osd.6 0 log_to_monitors true 2024-04-15T15:23:40.611 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-04-15T15:23:40.610+0000 7f22db400640 -1 osd.2 0 waiting for initial osdmap 2024-04-15T15:23:40.643 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-04-15T15:23:40.641+0000 7f22d1400640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:23:42.094 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-04-15T15:23:42.092+0000 7f8adfc00640 -1 osd.0 0 waiting for initial osdmap 2024-04-15T15:23:42.101 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-04-15T15:23:42.100+0000 7fe2bee00640 -1 osd.1 0 waiting for initial osdmap 2024-04-15T15:23:42.102 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-04-15T15:23:42.100+0000 7f8ad5c00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:23:42.109 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-04-15T15:23:42.108+0000 7fe2b4e00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:23:42.123 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-04-15T15:23:42.121+0000 7fcf4d600640 -1 osd.3 0 waiting for initial osdmap 2024-04-15T15:23:42.133 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-04-15T15:23:42.132+0000 7fcf42c00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:23:42.285 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:42.467 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.466+0000 7f70cc000640 1 Processor -- start 2024-04-15T15:23:42.468 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.466+0000 7f70cc000640 1 -- start start 2024-04-15T15:23:42.469 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.467+0000 7f70cc000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f70c4146370 0x7f70c406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:42.469 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.467+0000 7f70cc000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70c4072900 0x7f70c415d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:42.469 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70cc000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c415ce50 con 0x7f70c406e790 2024-04-15T15:23:42.470 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70cc000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f70c40a9630 con 0x7f70c4072900 2024-04-15T15:23:42.470 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70cc000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f70c406eca0 con 0x7f70c4146370 2024-04-15T15:23:42.470 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70ca000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70c4072900 0x7f70c415d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:42.470 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70c8c00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f70c406e790 0x7f70c40721f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:40608/0 (socket says 172.21.15.150:40608) 2024-04-15T15:23:42.470 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70c8c00640 1 -- 172.21.15.150:0/2737545776 learned_addr learned my addr 172.21.15.150:0/2737545776 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:42.470 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70ca000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70c4072900 0x7f70c415d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43972/0 (socket says 172.21.15.150:43972) 2024-04-15T15:23:42.470 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70c9600640 1 --2- 172.21.15.150:0/2737545776 >> v2:172.21.15.187:3300/0 conn(0x7f70c4146370 0x7f70c406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:42.471 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70ca000640 1 -- 172.21.15.150:0/2737545776 >> v2:172.21.15.187:3300/0 conn(0x7f70c4146370 msgr2=0x7f70c406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:42.471 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70ca000640 1 --2- 172.21.15.150:0/2737545776 >> v2:172.21.15.187:3300/0 conn(0x7f70c4146370 0x7f70c406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.471 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70ca000640 1 -- 172.21.15.150:0/2737545776 >> v1:172.21.15.150:6789/0 conn(0x7f70c406e790 legacy=0x7f70c40721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:42.471 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70ca000640 1 -- 172.21.15.150:0/2737545776 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c406ef80 con 0x7f70c4072900 2024-04-15T15:23:42.471 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.468+0000 7f70c9600640 1 --2- 172.21.15.150:0/2737545776 >> v2:172.21.15.187:3300/0 conn(0x7f70c4146370 0x7f70c406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:42.471 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.469+0000 7f70ca000640 1 --2- 172.21.15.150:0/2737545776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c4072900 0x7f70c415d8d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b5d20 tx=0x7f70bc02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b48901aa4b7e6db server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:42.472 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70c3e00640 1 -- 172.21.15.150:0/2737545776 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc039070 con 0x7f70c4072900 2024-04-15T15:23:42.472 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70c3e00640 1 -- 172.21.15.150:0/2737545776 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70bc02cc70 con 0x7f70c4072900 2024-04-15T15:23:42.472 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70c3e00640 1 -- 172.21.15.150:0/2737545776 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc034660 con 0x7f70c4072900 2024-04-15T15:23:42.472 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70cc000640 1 -- 172.21.15.150:0/2737545776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c4072900 msgr2=0x7f70c415d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70cc000640 1 --2- 172.21.15.150:0/2737545776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c4072900 0x7f70c415d8d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b5d20 tx=0x7f70bc02bec0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70cc000640 1 -- 172.21.15.150:0/2737545776 shutdown_connections 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70cc000640 1 --2- 172.21.15.150:0/2737545776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c4072900 0x7f70c415d8d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70cc000640 1 --2- 172.21.15.150:0/2737545776 >> v2:172.21.15.187:3300/0 conn(0x7f70c4146370 0x7f70c406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70cc000640 1 -- 172.21.15.150:0/2737545776 >> 172.21.15.150:0/2737545776 conn(0x7f70c40b25c0 msgr2=0x7f70c40b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70cc000640 1 -- 172.21.15.150:0/2737545776 shutdown_connections 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.470+0000 7f70cc000640 1 -- 172.21.15.150:0/2737545776 wait complete. 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.472+0000 7f70cc000640 1 Processor -- start 2024-04-15T15:23:42.473 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.472+0000 7f70cc000640 1 -- start start 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70cc000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c406e790 0x7f70c415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70cc000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70c4072900 0x7f70c40c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70cc000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70c4146370 0x7f70c40c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70cc000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70c406f850 con 0x7f70c406e790 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70cc000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70c406f6d0 con 0x7f70c4072900 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70cc000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70c415de10 con 0x7f70c4146370 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70c9600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c406e790 0x7f70c415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70c9600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c406e790 0x7f70c415c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43974/0 (socket says 172.21.15.150:43974) 2024-04-15T15:23:42.476 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70c9600640 1 -- 172.21.15.150:0/369946143 learned_addr learned my addr 172.21.15.150:0/369946143 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70c8c00640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70c4072900 0x7f70c40c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70ca000640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70c4146370 0x7f70c40c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70c9600640 1 -- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70c4146370 msgr2=0x7f70c40c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.473+0000 7f70c9600640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70c4146370 0x7f70c40c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70c9600640 1 -- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70c4072900 msgr2=0x7f70c40c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70c9600640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70c4072900 0x7f70c40c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70c9600640 1 -- 172.21.15.150:0/369946143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c415c9d0 con 0x7f70c406e790 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70ca000640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70c4146370 0x7f70c40c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70c9600640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c406e790 0x7f70c415c490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c7f0 tx=0x7f70b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70c1600640 1 -- 172.21.15.150:0/369946143 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8015070 con 0x7f70c406e790 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70c1600640 1 -- 172.21.15.150:0/369946143 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b80077c0 con 0x7f70c406e790 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70c1600640 1 -- 172.21.15.150:0/369946143 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8007b70 con 0x7f70c406e790 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c40c7550 con 0x7f70c406e790 2024-04-15T15:23:42.477 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.474+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c40c6790 con 0x7f70c406e790 2024-04-15T15:23:42.478 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.475+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c4068f30 con 0x7f70c406e790 2024-04-15T15:23:42.478 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.476+0000 7f70c1600640 1 -- 172.21.15.150:0/369946143 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f70b8018070 con 0x7f70c406e790 2024-04-15T15:23:42.479 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.477+0000 7f70c1600640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70a406c890 0x7f70a406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:42.479 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.477+0000 7f70c1600640 1 -- 172.21.15.150:0/369946143 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f70b808aec0 con 0x7f70c406e790 2024-04-15T15:23:42.479 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.477+0000 7f70c8c00640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70a406c890 0x7f70a406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:42.480 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.479+0000 7f70c8c00640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70a406c890 0x7f70a406ed50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b5ba0 tx=0x7f70b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:42.484 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.483+0000 7f70c1600640 1 -- 172.21.15.150:0/369946143 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70b8059fb0 con 0x7f70c406e790 2024-04-15T15:23:42.492 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-15T15:23:42.490+0000 7f4b8e800640 -1 osd.6 0 waiting for initial osdmap 2024-04-15T15:23:42.504 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-15T15:23:42.502+0000 7f4b84800640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:23:42.561 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-15T15:23:42.559+0000 7f442e800640 -1 osd.7 0 waiting for initial osdmap 2024-04-15T15:23:42.567 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-15T15:23:42.565+0000 7f4423e00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:23:42.578 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-15T15:23:42.576+0000 7fd841200640 -1 osd.5 0 waiting for initial osdmap 2024-04-15T15:23:42.586 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-15T15:23:42.584+0000 7fe29aa00640 -1 osd.4 0 waiting for initial osdmap 2024-04-15T15:23:42.588 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-15T15:23:42.586+0000 7fd836800640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:23:42.595 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-15T15:23:42.593+0000 7fe290a00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-15T15:23:42.687 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.685+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70c4073c30 con 0x7f70c406e790 2024-04-15T15:23:42.688 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.686+0000 7f70c1600640 1 -- 172.21.15.150:0/369946143 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7567 (secure 0 0 0) 0x7f70b80598b0 con 0x7f70c406e790 2024-04-15T15:23:42.689 INFO:teuthology.misc.health.smithi150.stdout: 2024-04-15T15:23:42.689 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":14,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:41.647141+0000","last_up_change":"2024-04-15T15:23:41.647141+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:42.691 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70a406c890 msgr2=0x7f70a406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:42.691 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70a406c890 0x7f70a406ed50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b5ba0 tx=0x7f70b0009040 comp rx=0 tx=0).stop 2024-04-15T15:23:42.691 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c406e790 msgr2=0x7f70c415c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:42.691 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c406e790 0x7f70c415c490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c7f0 tx=0x7f70b800ccb0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.692 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 shutdown_connections 2024-04-15T15:23:42.692 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70a406c890 0x7f70a406ed50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.692 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70c4146370 0x7f70c40c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.692 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70c4072900 0x7f70c40c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.692 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 --2- 172.21.15.150:0/369946143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c406e790 0x7f70c415c490 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:42.692 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.689+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 >> 172.21.15.150:0/369946143 conn(0x7f70c40b25c0 msgr2=0x7f70c40b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:42.692 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.690+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 shutdown_connections 2024-04-15T15:23:42.692 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:42.690+0000 7f70cc000640 1 -- 172.21.15.150:0/369946143 wait complete. 2024-04-15T15:23:42.704 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-15T15:23:46.379 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:23:46.384 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:23:49.511 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:49.698 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.696+0000 7fd91e800640 1 Processor -- start 2024-04-15T15:23:49.698 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.696+0000 7fd91e800640 1 -- start start 2024-04-15T15:23:49.699 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.698+0000 7fd91e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9180b5a30 0x7fd9180ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.700 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.698+0000 7fd91e800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd918145ae0 0x7fd9180b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.700 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.698+0000 7fd91e800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9180b6350 con 0x7fd9180722b0 2024-04-15T15:23:49.700 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.698+0000 7fd91e800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd91815cf80 con 0x7fd918145ae0 2024-04-15T15:23:49.700 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.698+0000 7fd91e800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd91815d260 con 0x7fd9180b5a30 2024-04-15T15:23:49.700 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.698+0000 7fd91de00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd9180722b0 0x7fd918072720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45846/0 (socket says 172.21.15.150:45846) 2024-04-15T15:23:49.700 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.698+0000 7fd91de00640 1 -- 172.21.15.150:0/85543444 learned_addr learned my addr 172.21.15.150:0/85543444 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:49.702 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.698+0000 7fd91ca00640 1 --2- 172.21.15.150:0/85543444 >> v2:172.21.15.187:3300/0 conn(0x7fd918145ae0 0x7fd9180b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:49.702 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd91d400640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9180b5a30 0x7fd9180ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:49.702 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3583887354 0 0) 0x7fd9180b6350 con 0x7fd9180722b0 2024-04-15T15:23:49.702 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd900003520 con 0x7fd9180722b0 2024-04-15T15:23:49.702 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1103836193 0 0) 0x7fd900003520 con 0x7fd9180722b0 2024-04-15T15:23:49.702 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9180b5a30 msgr2=0x7fd9180ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:49.702 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9180b5a30 0x7fd9180ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.702 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 >> v2:172.21.15.187:3300/0 conn(0x7fd918145ae0 msgr2=0x7fd9180b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 --2- 172.21.15.150:0/85543444 >> v2:172.21.15.187:3300/0 conn(0x7fd918145ae0 0x7fd9180b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91815d540 con 0x7fd9180722b0 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fd9100019f0 con 0x7fd9180722b0 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 >> v1:172.21.15.150:6789/0 conn(0x7fd9180722b0 legacy=0x7fd918072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000039b0 0x7fd900003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9180b5a30 0x7fd900004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd900004d40 0x7fd9000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd91815cf80 con 0x7fd900004d40 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd91815d260 con 0x7fd9000039b0 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.699+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd90000ba40 con 0x7fd9180b5a30 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.700+0000 7fd91de00640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd900004d40 0x7fd9000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.700+0000 7fd91d400640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000039b0 0x7fd900003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.700+0000 7fd91de00640 1 -- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9180b5a30 msgr2=0x7fd900004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:49.703 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.700+0000 7fd91de00640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9180b5a30 0x7fd900004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.704 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.700+0000 7fd91de00640 1 -- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000039b0 msgr2=0x7fd900003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:49.704 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.700+0000 7fd91de00640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000039b0 0x7fd900003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.704 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.700+0000 7fd91de00640 1 -- 172.21.15.150:0/85543444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd900003520 con 0x7fd900004d40 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.701+0000 7fd91de00640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd900004d40 0x7fd9000051c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd91000a870 tx=0x7fd900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dcd6bdebbb09f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.701+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91000a4c0 con 0x7fd900004d40 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.701+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd910034630 con 0x7fd900004d40 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.701+0000 7fd917e00640 1 -- 172.21.15.150:0/85543444 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910034a20 con 0x7fd900004d40 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.702+0000 7fd91e800640 1 -- 172.21.15.150:0/85543444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd900004d40 msgr2=0x7fd9000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.702+0000 7fd91e800640 1 --2- 172.21.15.150:0/85543444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd900004d40 0x7fd9000051c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd91000a870 tx=0x7fd900005830 comp rx=0 tx=0).stop 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.702+0000 7fd91de00640 1 -- 172.21.15.150:0/85543444 reap_dead start 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.702+0000 7fd91e800640 1 -- 172.21.15.150:0/85543444 shutdown_connections 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.702+0000 7fd91e800640 1 -- 172.21.15.150:0/85543444 >> 172.21.15.150:0/85543444 conn(0x7fd91806bd30 msgr2=0x7fd91806c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.702+0000 7fd91e800640 1 -- 172.21.15.150:0/85543444 shutdown_connections 2024-04-15T15:23:49.705 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.702+0000 7fd91e800640 1 -- 172.21.15.150:0/85543444 wait complete. 2024-04-15T15:23:49.706 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.704+0000 7fd91e800640 1 Processor -- start 2024-04-15T15:23:49.706 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.704+0000 7fd91e800640 1 -- start start 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd900004d40 0x7fd9180c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9181f6df0 0x7fd9181f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91e800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91813f2f0 0x7fd91813f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91e800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd91815cba0 con 0x7fd9181f6df0 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91e800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd91815db00 con 0x7fd91813f2f0 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91e800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd91815cd20 con 0x7fd900004d40 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9181f6df0 0x7fd9181f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9181f6df0 0x7fd9181f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59086/0 (socket says 172.21.15.150:59086) 2024-04-15T15:23:49.707 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.705+0000 7fd91ca00640 1 -- 172.21.15.150:0/3741826177 learned_addr learned my addr 172.21.15.150:0/3741826177 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:49.708 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91d400640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd900004d40 0x7fd9180c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91de00640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91813f2f0 0x7fd91813f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91ca00640 1 -- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd900004d40 msgr2=0x7fd9180c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91ca00640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd900004d40 0x7fd9180c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91ca00640 1 -- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91813f2f0 msgr2=0x7fd91813f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91ca00640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91813f2f0 0x7fd91813f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91ca00640 1 -- 172.21.15.150:0/3741826177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91813ffb0 con 0x7fd9181f6df0 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91de00640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91813f2f0 0x7fd91813f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91ca00640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9181f6df0 0x7fd9181f7270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e7c0 tx=0x7fd90400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd915600640 1 -- 172.21.15.150:0/3741826177 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904017070 con 0x7fd9181f6df0 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd915600640 1 -- 172.21.15.150:0/3741826177 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd90400c940 con 0x7fd9181f6df0 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd915600640 1 -- 172.21.15.150:0/3741826177 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90400cd30 con 0x7fd9181f6df0 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.706+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd918140290 con 0x7fd9181f6df0 2024-04-15T15:23:49.709 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.707+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd918140850 con 0x7fd9181f6df0 2024-04-15T15:23:49.710 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.708+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91804fd50 con 0x7fd9181f6df0 2024-04-15T15:23:49.710 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.709+0000 7fd915600640 1 -- 172.21.15.150:0/3741826177 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd90401a070 con 0x7fd9181f6df0 2024-04-15T15:23:49.711 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.710+0000 7fd915600640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8f806c8f0 0x7fd8f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:49.711 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.710+0000 7fd915600640 1 -- 172.21.15.150:0/3741826177 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fd90408ba70 con 0x7fd9181f6df0 2024-04-15T15:23:49.712 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.710+0000 7fd91d400640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8f806c8f0 0x7fd8f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:49.713 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.712+0000 7fd91d400640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8f806c8f0 0x7fd8f806edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009a10 tx=0x7fd90c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:49.716 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.715+0000 7fd915600640 1 -- 172.21.15.150:0/3741826177 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd90405a1f0 con 0x7fd9181f6df0 2024-04-15T15:23:49.919 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.918+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd918144b10 con 0x7fd9181f6df0 2024-04-15T15:23:49.922 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.920+0000 7fd915600640 1 -- 172.21.15.150:0/3741826177 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12932 (secure 0 0 0) 0x7fd904059af0 con 0x7fd9181f6df0 2024-04-15T15:23:49.922 INFO:teuthology.misc.health.smithi150.stdout: 2024-04-15T15:23:49.922 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":18,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:46.133083+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:49.925 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8f806c8f0 msgr2=0x7fd8f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:49.925 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8f806c8f0 0x7fd8f806edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009a10 tx=0x7fd90c008040 comp rx=0 tx=0).stop 2024-04-15T15:23:49.925 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9181f6df0 msgr2=0x7fd9181f7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:49.925 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9181f6df0 0x7fd9181f7270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e7c0 tx=0x7fd90400ec80 comp rx=0 tx=0).stop 2024-04-15T15:23:49.925 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 shutdown_connections 2024-04-15T15:23:49.925 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd91813f2f0 0x7fd91813f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.925 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9181f6df0 0x7fd9181f7270 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.925 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8f806c8f0 0x7fd8f806edb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.926 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 --2- 172.21.15.150:0/3741826177 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd900004d40 0x7fd9180c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:49.926 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 >> 172.21.15.150:0/3741826177 conn(0x7fd91806bd30 msgr2=0x7fd9181430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:49.926 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 shutdown_connections 2024-04-15T15:23:49.926 INFO:teuthology.misc.health.smithi150.stderr:2024-04-15T15:23:49.923+0000 7fd91e800640 1 -- 172.21.15.150:0/3741826177 wait complete. 2024-04-15T15:23:49.940 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-15T15:23:49.940 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-15T15:23:49.940 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-15T15:23:49.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-15T15:23:49.947 INFO:tasks.ceph.mds.a:Started 2024-04-15T15:23:49.947 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-15T15:23:49.947 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-15T15:23:49.955 INFO:tasks.ceph.mds.c:Started 2024-04-15T15:23:49.955 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-15T15:23:49.955 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-15T15:23:49.963 INFO:tasks.ceph.mds.b:Started 2024-04-15T15:23:49.963 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-15T15:23:49.963 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-15T15:23:49.970 INFO:tasks.ceph.mds.d:Started 2024-04-15T15:23:49.970 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-15T15:23:49.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:50.101 INFO:tasks.ceph.mds.a.smithi150.stdout:starting mds.a at 2024-04-15T15:23:50.103 INFO:tasks.ceph.mds.c.smithi150.stdout:starting mds.c at 2024-04-15T15:23:50.132 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-15T15:23:50.132 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-15T15:23:50.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.155+0000 7ff36fe00640 1 Processor -- start 2024-04-15T15:23:50.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.155+0000 7ff36fe00640 1 -- start start 2024-04-15T15:23:50.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.156+0000 7ff36fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff36815edb0 0x7ff36809ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff3680ce860 0x7ff36809b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36fe00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff36806cd90 con 0x7ff3680b3ec0 2024-04-15T15:23:50.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36fe00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7ff3680cce20 con 0x7ff36815edb0 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff36806d070 con 0x7ff3680ce860 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36ca00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7ff3680b3ec0 0x7ff3680c7b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45856/0 (socket says 172.21.15.150:45856) 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36ca00640 1 -- 172.21.15.150:0/654246926 learned_addr learned my addr 172.21.15.150:0/654246926 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36d400640 1 --2- 172.21.15.150:0/654246926 >> v2:172.21.15.150:3300/0 conn(0x7ff36815edb0 0x7ff36809ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36de00640 1 --2- 172.21.15.150:0/654246926 >> v2:172.21.15.187:3300/0 conn(0x7ff3680ce860 0x7ff36809b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36d400640 1 -- 172.21.15.150:0/654246926 >> v2:172.21.15.187:3300/0 conn(0x7ff3680ce860 msgr2=0x7ff36809b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36d400640 1 --2- 172.21.15.150:0/654246926 >> v2:172.21.15.187:3300/0 conn(0x7ff3680ce860 0x7ff36809b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36d400640 1 -- 172.21.15.150:0/654246926 >> v1:172.21.15.150:6789/0 conn(0x7ff3680b3ec0 legacy=0x7ff3680c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff367e00640 1 -- 172.21.15.150:0/654246926 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2512596553 0 0) 0x7ff36806cd90 con 0x7ff3680b3ec0 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.157+0000 7ff36d400640 1 -- 172.21.15.150:0/654246926 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3680cd070 con 0x7ff36815edb0 2024-04-15T15:23:50.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.158+0000 7ff36d400640 1 --2- 172.21.15.150:0/654246926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff36815edb0 0x7ff36809ab90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff3680d2780 tx=0x7ff35c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157c05182c755a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:50.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff367e00640 1 -- 172.21.15.150:0/654246926 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c035020 con 0x7ff36815edb0 2024-04-15T15:23:50.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff367e00640 1 -- 172.21.15.150:0/654246926 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff35c031830 con 0x7ff36815edb0 2024-04-15T15:23:50.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff367e00640 1 -- 172.21.15.150:0/654246926 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c031c00 con 0x7ff36815edb0 2024-04-15T15:23:50.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff36fe00640 1 -- 172.21.15.150:0/654246926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff36815edb0 msgr2=0x7ff36809ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff36fe00640 1 --2- 172.21.15.150:0/654246926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff36815edb0 0x7ff36809ab90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff3680d2780 tx=0x7ff35c02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff36fe00640 1 -- 172.21.15.150:0/654246926 shutdown_connections 2024-04-15T15:23:50.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff36fe00640 1 --2- 172.21.15.150:0/654246926 >> v2:172.21.15.187:3300/0 conn(0x7ff3680ce860 0x7ff36809b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff36fe00640 1 --2- 172.21.15.150:0/654246926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff36815edb0 0x7ff36809ab90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.159+0000 7ff36fe00640 1 -- 172.21.15.150:0/654246926 >> 172.21.15.150:0/654246926 conn(0x7ff3680d27d0 msgr2=0x7ff3680d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:50.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.160+0000 7ff36fe00640 1 -- 172.21.15.150:0/654246926 shutdown_connections 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.160+0000 7ff36fe00640 1 -- 172.21.15.150:0/654246926 wait complete. 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.161+0000 7ff36fe00640 1 Processor -- start 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.161+0000 7ff36fe00640 1 -- start start 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3680b3ec0 0x7ff3680b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680ce860 0x7ff36815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff36815edb0 0x7ff3681576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff36806d450 con 0x7ff3680b3ec0 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff36806ca40 con 0x7ff36815edb0 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff36806d5d0 con 0x7ff3680ce860 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3680b3ec0 0x7ff3680b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3680b3ec0 0x7ff3680b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59140/0 (socket says 172.21.15.150:59140) 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.162+0000 7ff36d400640 1 -- 172.21.15.150:0/1202021201 learned_addr learned my addr 172.21.15.150:0/1202021201 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.163+0000 7ff36de00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff36815edb0 0x7ff3681576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.163+0000 7ff36ca00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680ce860 0x7ff36815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.163+0000 7ff36d400640 1 -- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680ce860 msgr2=0x7ff36815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.163+0000 7ff36d400640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680ce860 0x7ff36815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.163+0000 7ff36d400640 1 -- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff36815edb0 msgr2=0x7ff3681576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.163+0000 7ff36d400640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff36815edb0 0x7ff3681576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.163+0000 7ff36d400640 1 -- 172.21.15.150:0/1202021201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3680b66c0 con 0x7ff3680b3ec0 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.163+0000 7ff36d400640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3680b3ec0 0x7ff3680b7b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009890 tx=0x7ff35c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.164+0000 7ff365600640 1 -- 172.21.15.150:0/1202021201 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c041070 con 0x7ff3680b3ec0 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.164+0000 7ff365600640 1 -- 172.21.15.150:0/1202021201 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff35c03aa30 con 0x7ff3680b3ec0 2024-04-15T15:23:50.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.164+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3680b69a0 con 0x7ff3680b3ec0 2024-04-15T15:23:50.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.164+0000 7ff365600640 1 -- 172.21.15.150:0/1202021201 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c035040 con 0x7ff3680b3ec0 2024-04-15T15:23:50.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.164+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3680b6f60 con 0x7ff3680b3ec0 2024-04-15T15:23:50.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.165+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff368068c80 con 0x7ff3680b3ec0 2024-04-15T15:23:50.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.166+0000 7ff365600640 1 -- 172.21.15.150:0/1202021201 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff35c038020 con 0x7ff3680b3ec0 2024-04-15T15:23:50.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.167+0000 7ff365600640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff34806c970 0x7ff34806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.167+0000 7ff365600640 1 -- 172.21.15.150:0/1202021201 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7ff35c0ad170 con 0x7ff3680b3ec0 2024-04-15T15:23:50.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.167+0000 7ff36ca00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff34806c970 0x7ff34806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.168+0000 7ff36ca00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff34806c970 0x7ff34806ee30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff368162190 tx=0x7ff354007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:50.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.174+0000 7ff365600640 1 -- 172.21.15.150:0/1202021201 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff35c07b280 con 0x7ff3680b3ec0 2024-04-15T15:23:50.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.372+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff368160fe0 con 0x7ff3680b3ec0 2024-04-15T15:23:50.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.374+0000 7ff365600640 1 -- 172.21.15.150:0/1202021201 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12932 (secure 0 0 0) 0x7ff35c07f580 con 0x7ff3680b3ec0 2024-04-15T15:23:50.376 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:50.376 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":18,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:46.133083+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:50.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff34806c970 msgr2=0x7ff34806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff34806c970 0x7ff34806ee30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff368162190 tx=0x7ff354007de0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3680b3ec0 msgr2=0x7ff3680b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3680b3ec0 0x7ff3680b7b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009890 tx=0x7ff35c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 shutdown_connections 2024-04-15T15:23:50.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff36815edb0 0x7ff3681576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff34806c970 0x7ff34806ee30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680ce860 0x7ff36815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1202021201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3680b3ec0 0x7ff3680b7b90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 >> 172.21.15.150:0/1202021201 conn(0x7ff3680d27d0 msgr2=0x7ff3680d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:50.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 shutdown_connections 2024-04-15T15:23:50.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.377+0000 7ff36fe00640 1 -- 172.21.15.150:0/1202021201 wait complete. 2024-04-15T15:23:50.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:23:50.395 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:23:50.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.586+0000 7f9931e00640 1 Processor -- start 2024-04-15T15:23:50.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.586+0000 7f9931e00640 1 -- start start 2024-04-15T15:23:50.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.587+0000 7f9931e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c15d510 0x7f992c06ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.587+0000 7f9931e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f992c15d8f0 0x7f992c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f9931e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f992c0cecd0 con 0x7f992c0ce850 2024-04-15T15:23:50.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f9931e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f992c0cefb0 con 0x7f992c15d8f0 2024-04-15T15:23:50.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f9931e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f992c06dfa0 con 0x7f992c15d510 2024-04-15T15:23:50.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f992be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f992c15d8f0 0x7f992c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f992be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f992c15d8f0 0x7f992c0ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59146/0 (socket says 172.21.15.150:59146) 2024-04-15T15:23:50.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f992be00640 1 -- 172.21.15.150:0/3579317239 learned_addr learned my addr 172.21.15.150:0/3579317239 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:50.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f992be00640 1 -- 172.21.15.150:0/3579317239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c15d510 msgr2=0x7f992c06ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:50.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f992be00640 1 --2- 172.21.15.150:0/3579317239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c15d510 0x7f992c06ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f992be00640 1 -- 172.21.15.150:0/3579317239 >> v1:172.21.15.187:6789/0 conn(0x7f992c0ce850 legacy=0x7f992c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:50.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.588+0000 7f992be00640 1 -- 172.21.15.150:0/3579317239 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c06e280 con 0x7f992c15d8f0 2024-04-15T15:23:50.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.589+0000 7f992be00640 1 --2- 172.21.15.150:0/3579317239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d8f0 0x7f992c0ce310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9918009870 tx=0x7f991802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7034493a3097941a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.589+0000 7f992b400640 1 -- 172.21.15.150:0/3579317239 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918035020 con 0x7f992c15d8f0 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f992b400640 1 -- 172.21.15.150:0/3579317239 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9918031640 con 0x7f992c15d8f0 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f992b400640 1 -- 172.21.15.150:0/3579317239 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991802d040 con 0x7f992c15d8f0 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f9931e00640 1 -- 172.21.15.150:0/3579317239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d8f0 msgr2=0x7f992c0ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f9931e00640 1 --2- 172.21.15.150:0/3579317239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d8f0 0x7f992c0ce310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9918009870 tx=0x7f991802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f9931e00640 1 -- 172.21.15.150:0/3579317239 shutdown_connections 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f9931e00640 1 --2- 172.21.15.150:0/3579317239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d8f0 0x7f992c0ce310 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f9931e00640 1 --2- 172.21.15.150:0/3579317239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c15d510 0x7f992c06ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f9931e00640 1 -- 172.21.15.150:0/3579317239 >> 172.21.15.150:0/3579317239 conn(0x7f992c0b23a0 msgr2=0x7f992c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:50.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f9931e00640 1 -- 172.21.15.150:0/3579317239 shutdown_connections 2024-04-15T15:23:50.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.590+0000 7f9931e00640 1 -- 172.21.15.150:0/3579317239 wait complete. 2024-04-15T15:23:50.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.592+0000 7f9931e00640 1 Processor -- start 2024-04-15T15:23:50.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.592+0000 7f9931e00640 1 -- start start 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f9931e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f992c0ce850 0x7f992c0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f9931e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d510 0x7f992c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f9931e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c158690 0x7f992c158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f9931e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f992c06efc0 con 0x7f992c15d510 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f9931e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f992c0b60c0 con 0x7f992c0ce850 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f9931e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f992c14b990 con 0x7f992c158690 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f992be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d510 0x7f992c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f992be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d510 0x7f992c0c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59148/0 (socket says 172.21.15.150:59148) 2024-04-15T15:23:50.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.593+0000 7f992be00640 1 -- 172.21.15.150:0/2317563970 learned_addr learned my addr 172.21.15.150:0/2317563970 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f9930a00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f992c0ce850 0x7f992c0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f992be00640 1 -- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c158690 msgr2=0x7f992c158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f9931400640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c158690 0x7f992c158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f992be00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c158690 0x7f992c158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f992be00640 1 -- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f992c0ce850 msgr2=0x7f992c0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f992be00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f992c0ce850 0x7f992c0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f992be00640 1 -- 172.21.15.150:0/2317563970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c14bc70 con 0x7f992c15d510 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f9930a00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f992c0ce850 0x7f992c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f992be00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d510 0x7f992c0c9620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f991802b930 tx=0x7f99180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f9928c00640 1 -- 172.21.15.150:0/2317563970 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991803ac60 con 0x7f992c15d510 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f9928c00640 1 -- 172.21.15.150:0/2317563970 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991803ae00 con 0x7f992c15d510 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f9928c00640 1 -- 172.21.15.150:0/2317563970 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918044630 con 0x7f992c15d510 2024-04-15T15:23:50.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.594+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c14bf50 con 0x7f992c15d510 2024-04-15T15:23:50.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.595+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c0c9b60 con 0x7f992c15d510 2024-04-15T15:23:50.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.596+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992c0755c0 con 0x7f992c15d510 2024-04-15T15:23:50.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.596+0000 7f9928c00640 1 -- 172.21.15.150:0/2317563970 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f991804b060 con 0x7f992c15d510 2024-04-15T15:23:50.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.597+0000 7f9928c00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f990c06c8f0 0x7f990c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:50.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.597+0000 7f9928c00640 1 -- 172.21.15.150:0/2317563970 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f99180ad610 con 0x7f992c15d510 2024-04-15T15:23:50.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.598+0000 7f9930a00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f990c06c8f0 0x7f990c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:50.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.599+0000 7f9930a00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f990c06c8f0 0x7f990c06edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9920006fd0 tx=0x7f9920008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:50.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.602+0000 7f9928c00640 1 -- 172.21.15.150:0/2317563970 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f991803f070 con 0x7f992c15d510 2024-04-15T15:23:50.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.800+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f992c15a230 con 0x7f992c15d510 2024-04-15T15:23:50.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.801+0000 7f9928c00640 1 -- 172.21.15.150:0/2317563970 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7f991807bcf0 con 0x7f992c15d510 2024-04-15T15:23:50.803 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:23:50.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.803+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f990c06c8f0 msgr2=0x7f990c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.803+0000 7f9931e00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f990c06c8f0 0x7f990c06edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9920006fd0 tx=0x7f9920008040 comp rx=0 tx=0).stop 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.803+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d510 msgr2=0x7f992c0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.803+0000 7f9931e00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d510 0x7f992c0c9620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f991802b930 tx=0x7f99180046c0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.804+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 shutdown_connections 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.804+0000 7f9931e00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f990c06c8f0 0x7f990c06edb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.804+0000 7f9931e00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f992c158690 0x7f992c158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.804+0000 7f9931e00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f992c15d510 0x7f992c0c9620 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.804+0000 7f9931e00640 1 --2- 172.21.15.150:0/2317563970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f992c0ce850 0x7f992c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.804+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 >> 172.21.15.150:0/2317563970 conn(0x7f992c0b23a0 msgr2=0x7f992c0b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.804+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 shutdown_connections 2024-04-15T15:23:50.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:50.804+0000 7f9931e00640 1 -- 172.21.15.150:0/2317563970 wait complete. 2024-04-15T15:23:50.823 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:23:51.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.025+0000 7f63b8c00640 1 Processor -- start 2024-04-15T15:23:51.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.025+0000 7f63b8c00640 1 -- start start 2024-04-15T15:23:51.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.027+0000 7f63b8c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f63b40d03b0 0x7f63b4144de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.027+0000 7f63b8c00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b40d0b20 con 0x7f63b415eba0 2024-04-15T15:23:51.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.027+0000 7f63b8c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b40d0e00 con 0x7f63b40b3290 2024-04-15T15:23:51.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.027+0000 7f63b8c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f63b40d13c0 con 0x7f63b40d03b0 2024-04-15T15:23:51.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.027+0000 7f63b3400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f63b415eba0 0x7f63b415bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45858/0 (socket says 172.21.15.150:45858) 2024-04-15T15:23:51.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.027+0000 7f63b3400640 1 -- 172.21.15.150:0/57360822 learned_addr learned my addr 172.21.15.150:0/57360822 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:51.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2333971651 0 0) 0x7f63b40d0b20 con 0x7f63b415eba0 2024-04-15T15:23:51.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6398003520 con 0x7f63b415eba0 2024-04-15T15:23:51.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63aaa00640 1 --2- 172.21.15.150:0/57360822 >> v2:172.21.15.187:3300/0 conn(0x7f63b40d03b0 0x7f63b4144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 950897780 0 0) 0x7f6398003520 con 0x7f63b415eba0 2024-04-15T15:23:51.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 >> v2:172.21.15.187:3300/0 conn(0x7f63b40d03b0 msgr2=0x7f63b4144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 --2- 172.21.15.150:0/57360822 >> v2:172.21.15.187:3300/0 conn(0x7f63b40d03b0 0x7f63b4144de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 >> v1:172.21.15.187:6789/0 conn(0x7f63b40b3290 legacy=0x7f63b40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b40d10e0 con 0x7f63b415eba0 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f63a4001670 con 0x7f63b415eba0 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 >> v1:172.21.15.150:6789/0 conn(0x7f63b415eba0 legacy=0x7f63b415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63980039b0 0x7f6398003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6398004800 0x7f6398004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980051c0 0x7f639800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f639800e2f0 con 0x7f63980039b0 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f639800e4a0 con 0x7f6398004800 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f639800e6a0 con 0x7f63980051c0 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.028+0000 7f63b3400640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63980039b0 0x7f6398003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63aaa00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6398004800 0x7f6398004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63b3e00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980051c0 0x7f639800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63b3400640 1 -- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980051c0 msgr2=0x7f639800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63b3400640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980051c0 0x7f639800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63b3400640 1 -- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6398004800 msgr2=0x7f6398004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63b3400640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6398004800 0x7f6398004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63b3400640 1 -- 172.21.15.150:0/57360822 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6398003520 con 0x7f63980039b0 2024-04-15T15:23:51.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63aaa00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6398004800 0x7f6398004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:51.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63b3e00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980051c0 0x7f639800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:51.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.029+0000 7f63b3e00640 1 -- 172.21.15.150:0/57360822 reap_dead start 2024-04-15T15:23:51.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.030+0000 7f63b3400640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63980039b0 0x7f6398003dd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f63a400c240 tx=0x7f6398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2714dbccfbfe85 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.030+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4007cf0 con 0x7f63980039b0 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.030+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a4030070 con 0x7f63980039b0 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.030+0000 7f63b2a00640 1 -- 172.21.15.150:0/57360822 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4005cf0 con 0x7f63980039b0 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.031+0000 7f63b8c00640 1 -- 172.21.15.150:0/57360822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63980039b0 msgr2=0x7f6398003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.031+0000 7f63b8c00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63980039b0 0x7f6398003dd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f63a400c240 tx=0x7f6398005830 comp rx=0 tx=0).stop 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.031+0000 7f63b8c00640 1 -- 172.21.15.150:0/57360822 shutdown_connections 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.031+0000 7f63b8c00640 1 --2- 172.21.15.150:0/57360822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63980039b0 0x7f6398003dd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.031+0000 7f63b8c00640 1 -- 172.21.15.150:0/57360822 >> 172.21.15.150:0/57360822 conn(0x7f63b4067720 msgr2=0x7f63b4067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.031+0000 7f63b8c00640 1 -- 172.21.15.150:0/57360822 shutdown_connections 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.031+0000 7f63b8c00640 1 -- 172.21.15.150:0/57360822 wait complete. 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.033+0000 7f63b8c00640 1 Processor -- start 2024-04-15T15:23:51.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.033+0000 7f63b8c00640 1 -- start start 2024-04-15T15:23:51.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980039b0 0x7f63b41f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b8c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b4138980 0x7f63b415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b8c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b40acbb0 0x7f63b40ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b8c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40b3700 con 0x7f63b40acbb0 2024-04-15T15:23:51.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b8c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40d0840 con 0x7f63b4138980 2024-04-15T15:23:51.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63b40b3880 con 0x7f63980039b0 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b40acbb0 0x7f63b40ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b40acbb0 0x7f63b40ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59168/0 (socket says 172.21.15.150:59168) 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3e00640 1 -- 172.21.15.150:0/3458856004 learned_addr learned my addr 172.21.15.150:0/3458856004 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63aaa00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b4138980 0x7f63b415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3400640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980039b0 0x7f63b41f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3e00640 1 -- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980039b0 msgr2=0x7f63b41f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3e00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980039b0 0x7f63b41f7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3e00640 1 -- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b4138980 msgr2=0x7f63b415ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3e00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b4138980 0x7f63b415ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3e00640 1 -- 172.21.15.150:0/3458856004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b415cb90 con 0x7f63b40acbb0 2024-04-15T15:23:51.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63aaa00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b4138980 0x7f63b415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:51.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.034+0000 7f63b3400640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980039b0 0x7f63b41f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:51.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.035+0000 7f63b3e00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b40acbb0 0x7f63b40ad060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63b40b43c0 tx=0x7f63ac00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.035+0000 7f63abe00640 1 -- 172.21.15.150:0/3458856004 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac018070 con 0x7f63b40acbb0 2024-04-15T15:23:51.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.035+0000 7f63abe00640 1 -- 172.21.15.150:0/3458856004 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac003da0 con 0x7f63b40acbb0 2024-04-15T15:23:51.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.035+0000 7f63abe00640 1 -- 172.21.15.150:0/3458856004 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0052b0 con 0x7f63b40acbb0 2024-04-15T15:23:51.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.035+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b4138e60 con 0x7f63b40acbb0 2024-04-15T15:23:51.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.035+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b4139420 con 0x7f63b40acbb0 2024-04-15T15:23:51.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.036+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b40755c0 con 0x7f63b40acbb0 2024-04-15T15:23:51.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.037+0000 7f63abe00640 1 -- 172.21.15.150:0/3458856004 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63ac005450 con 0x7f63b40acbb0 2024-04-15T15:23:51.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.038+0000 7f63abe00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f639006c8f0 0x7f639006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.038+0000 7f63abe00640 1 -- 172.21.15.150:0/3458856004 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f63ac08ab20 con 0x7f63b40acbb0 2024-04-15T15:23:51.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.039+0000 7f63b3400640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f639006c8f0 0x7f639006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.040+0000 7f63b3400640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f639006c8f0 0x7f639006edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63a4034000 tx=0x7f63a4007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.043+0000 7f63abe00640 1 -- 172.21.15.150:0/3458856004 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63ac0593f0 con 0x7f63b40acbb0 2024-04-15T15:23:51.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.271+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63b406e040 con 0x7f63b40acbb0 2024-04-15T15:23:51.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.272+0000 7f63abe00640 1 -- 172.21.15.150:0/3458856004 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7f63ac058cf0 con 0x7f63b40acbb0 2024-04-15T15:23:51.274 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:51.274 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-15T15:23:51.274 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 3 2024-04-15T15:23:51.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f639006c8f0 msgr2=0x7f639006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f639006c8f0 0x7f639006edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63a4034000 tx=0x7f63a4007000 comp rx=0 tx=0).stop 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b40acbb0 msgr2=0x7f63b40ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b40acbb0 0x7f63b40ad060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63b40b43c0 tx=0x7f63ac00d9d0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 shutdown_connections 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b40acbb0 0x7f63b40ad060 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b4138980 0x7f63b415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f639006c8f0 0x7f639006edb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 --2- 172.21.15.150:0/3458856004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63980039b0 0x7f63b41f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 >> 172.21.15.150:0/3458856004 conn(0x7f63b4067720 msgr2=0x7f63b4074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 shutdown_connections 2024-04-15T15:23:51.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.275+0000 7f63b8c00640 1 -- 172.21.15.150:0/3458856004 wait complete. 2024-04-15T15:23:51.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:51.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.494+0000 7fd855800640 1 Processor -- start 2024-04-15T15:23:51.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.494+0000 7fd855800640 1 -- start start 2024-04-15T15:23:51.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.496+0000 7fd855800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 0x7fd850162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.496+0000 7fd855800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8500d0090 con 0x7fd8500d1c90 2024-04-15T15:23:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.496+0000 7fd855800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8500cfad0 con 0x7fd850162020 2024-04-15T15:23:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.496+0000 7fd855800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd850163440 con 0x7fd8500cf110 2024-04-15T15:23:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.496+0000 7fd847400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd8500d1c90 0x7fd8500d3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45860/0 (socket says 172.21.15.150:45860) 2024-04-15T15:23:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.496+0000 7fd847400640 1 -- 172.21.15.150:0/2364571563 learned_addr learned my addr 172.21.15.150:0/2364571563 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416898552 0 0) 0x7fd838001490 con 0x7fd8500d1c90 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd854e00640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 0x7fd850162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd834003520 con 0x7fd8500d1c90 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3986500046 0 0) 0x7fd834003520 con 0x7fd8500d1c90 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 msgr2=0x7fd850162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 0x7fd850162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 >> v1:172.21.15.187:6789/0 conn(0x7fd850162020 legacy=0x7fd8500d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8500cfdb0 con 0x7fd8500d1c90 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fd838001bc0 con 0x7fd8500d1c90 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd854e00640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 0x7fd850162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 >> v1:172.21.15.150:6789/0 conn(0x7fd8500d1c90 legacy=0x7fd8500d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8340039e0 0x7fd834003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834004800 0x7fd834004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 0x7fd8340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd850163440 con 0x7fd834004800 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd83400bac0 con 0x7fd8340039e0 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd83400bcc0 con 0x7fd8500cf110 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.497+0000 7fd847400640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834004800 0x7fd834004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.498+0000 7fd854e00640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 0x7fd8340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.498+0000 7fd84fe00640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8340039e0 0x7fd834003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.498+0000 7fd847400640 1 -- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 msgr2=0x7fd8340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.498+0000 7fd847400640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8500cf110 0x7fd8340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.498+0000 7fd847400640 1 -- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8340039e0 msgr2=0x7fd834003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.498+0000 7fd847400640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8340039e0 0x7fd834003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.498+0000 7fd847400640 1 -- 172.21.15.150:0/2364571563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd834003520 con 0x7fd834004800 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.499+0000 7fd847400640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834004800 0x7fd834004c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd83800aa50 tx=0x7fd834005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fac8e7734c919a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.499+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838038070 con 0x7fd834004800 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.499+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd838033650 con 0x7fd834004800 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.499+0000 7fd84f400640 1 -- 172.21.15.150:0/2364571563 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83800a670 con 0x7fd834004800 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.500+0000 7fd855800640 1 -- 172.21.15.150:0/2364571563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834004800 msgr2=0x7fd834004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.500+0000 7fd855800640 1 --2- 172.21.15.150:0/2364571563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834004800 0x7fd834004c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd83800aa50 tx=0x7fd834005830 comp rx=0 tx=0).stop 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.500+0000 7fd854e00640 1 -- 172.21.15.150:0/2364571563 reap_dead start 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.500+0000 7fd855800640 1 -- 172.21.15.150:0/2364571563 shutdown_connections 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.500+0000 7fd855800640 1 -- 172.21.15.150:0/2364571563 >> 172.21.15.150:0/2364571563 conn(0x7fd85006cf00 msgr2=0x7fd85006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.500+0000 7fd855800640 1 -- 172.21.15.150:0/2364571563 shutdown_connections 2024-04-15T15:23:51.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.500+0000 7fd855800640 1 -- 172.21.15.150:0/2364571563 wait complete. 2024-04-15T15:23:51.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.502+0000 7fd855800640 1 Processor -- start 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.502+0000 7fd855800640 1 -- start start 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd855800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd834004800 0x7fd85014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd855800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd850160180 0x7fd8500c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd855800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8501517f0 0x7fd8501608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd855800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd8500cf880 con 0x7fd8501517f0 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd855800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8500b7b30 con 0x7fd834004800 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd855800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd8500d2e70 con 0x7fd850160180 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd854e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8501517f0 0x7fd8501608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd854e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8501517f0 0x7fd8501608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59188/0 (socket says 172.21.15.150:59188) 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd854e00640 1 -- 172.21.15.150:0/386968780 learned_addr learned my addr 172.21.15.150:0/386968780 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd847400640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd850160180 0x7fd8500c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd854e00640 1 -- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd850160180 msgr2=0x7fd8500c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd854e00640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd850160180 0x7fd8500c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd854e00640 1 -- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd834004800 msgr2=0x7fd85014ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd854e00640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd834004800 0x7fd85014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.503+0000 7fd854e00640 1 -- 172.21.15.150:0/386968780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8500b7d80 con 0x7fd8501517f0 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.504+0000 7fd854e00640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8501517f0 0x7fd8501608a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8500cf560 tx=0x7fd848010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.504+0000 7fd84cc00640 1 -- 172.21.15.150:0/386968780 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84800f040 con 0x7fd8501517f0 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.504+0000 7fd84cc00640 1 -- 172.21.15.150:0/386968780 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84801b070 con 0x7fd8501517f0 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.504+0000 7fd84cc00640 1 -- 172.21.15.150:0/386968780 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848003610 con 0x7fd8501517f0 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.504+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8500c8c50 con 0x7fd8501517f0 2024-04-15T15:23:51.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.504+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8500b82d0 con 0x7fd8501517f0 2024-04-15T15:23:51.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.505+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8500755c0 con 0x7fd8501517f0 2024-04-15T15:23:51.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.506+0000 7fd84cc00640 1 -- 172.21.15.150:0/386968780 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8480037b0 con 0x7fd8501517f0 2024-04-15T15:23:51.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.507+0000 7fd84cc00640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd82c06c920 0x7fd82c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.507+0000 7fd84cc00640 1 -- 172.21.15.150:0/386968780 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fd84808b650 con 0x7fd8501517f0 2024-04-15T15:23:51.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.508+0000 7fd84fe00640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd82c06c920 0x7fd82c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.509+0000 7fd84fe00640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd82c06c920 0x7fd82c06ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd840007fd0 tx=0x7fd840009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.512+0000 7fd84cc00640 1 -- 172.21.15.150:0/386968780 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd848059dc0 con 0x7fd8501517f0 2024-04-15T15:23:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.698+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8500b7650 con 0x7fd8501517f0 2024-04-15T15:23:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.700+0000 7fd84cc00640 1 -- 172.21.15.150:0/386968780 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12932 (secure 0 0 0) 0x7fd8480596c0 con 0x7fd8501517f0 2024-04-15T15:23:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:51.702 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":18,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:46.133083+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:51.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd82c06c920 msgr2=0x7fd82c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd82c06c920 0x7fd82c06ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd840007fd0 tx=0x7fd840009040 comp rx=0 tx=0).stop 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8501517f0 msgr2=0x7fd8501608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8501517f0 0x7fd8501608a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8500cf560 tx=0x7fd848010930 comp rx=0 tx=0).stop 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 shutdown_connections 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8501517f0 0x7fd8501608a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd82c06c920 0x7fd82c06ede0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd850160180 0x7fd8500c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 --2- 172.21.15.150:0/386968780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd834004800 0x7fd85014ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 >> 172.21.15.150:0/386968780 conn(0x7fd85006cf00 msgr2=0x7fd850150ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 shutdown_connections 2024-04-15T15:23:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.703+0000 7fd855800640 1 -- 172.21.15.150:0/386968780 wait complete. 2024-04-15T15:23:51.722 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:23:51.723 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:23:51.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.926+0000 7f7dd8600640 1 Processor -- start 2024-04-15T15:23:51.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.927+0000 7f7dd8600640 1 -- start start 2024-04-15T15:23:51.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.928+0000 7f7dd8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 0x7f7dd00d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.928+0000 7f7dd8600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd00ba4d0 con 0x7f7dd00d3270 2024-04-15T15:23:51.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.928+0000 7f7dd8600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd00ba7b0 con 0x7f7dd00b3720 2024-04-15T15:23:51.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.928+0000 7f7dd8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd015f0a0 con 0x7f7dd015ecc0 2024-04-15T15:23:51.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.928+0000 7f7dd5200640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7dd00d3270 0x7f7dd00b95f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45866/0 (socket says 172.21.15.150:45866) 2024-04-15T15:23:51.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.928+0000 7f7dd5200640 1 -- 172.21.15.150:0/980430821 learned_addr learned my addr 172.21.15.150:0/980430821 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:51.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dd5c00640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 0x7f7dd00d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819617774 0 0) 0x7f7dd00ba4d0 con 0x7f7dd00d3270 2024-04-15T15:23:51.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7db8003520 con 0x7f7dd00d3270 2024-04-15T15:23:51.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1563899411 0 0) 0x7f7db8003520 con 0x7f7dd00d3270 2024-04-15T15:23:51.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 msgr2=0x7f7dd00d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 0x7f7dd00d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 >> v1:172.21.15.187:6789/0 conn(0x7f7dd00b3720 legacy=0x7f7dd00cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd00b8df0 con 0x7f7dd00d3270 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f7dbc001670 con 0x7f7dd00d3270 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 >> v1:172.21.15.150:6789/0 conn(0x7f7dd00d3270 legacy=0x7f7dd00b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7db80039e0 0x7f7db8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7db8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 0x7f7db80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd015f0a0 con 0x7f7db8004800 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7db800ba70 con 0x7f7db80039e0 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.929+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7db800bc70 con 0x7f7dd015ecc0 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd5200640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7db8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd5c00640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7db80039e0 0x7f7db8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd6600640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 0x7f7db80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd5200640 1 -- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 msgr2=0x7f7db80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd5200640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 0x7f7db80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd5200640 1 -- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7db80039e0 msgr2=0x7f7db8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd5200640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7db80039e0 0x7f7db8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd5200640 1 -- 172.21.15.150:0/980430821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db8003520 con 0x7f7db8004800 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd5c00640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7db80039e0 0x7f7db8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.930+0000 7f7dd6600640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd015ecc0 0x7f7db80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.931+0000 7f7dd5200640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7db8004c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00c210 tx=0x7f7db8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1ca541f2d9e0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.931+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc007d10 con 0x7f7db8004800 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.931+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dbc030070 con 0x7f7db8004800 2024-04-15T15:23:51.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.931+0000 7f7dcfe00640 1 -- 172.21.15.150:0/980430821 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc005d50 con 0x7f7db8004800 2024-04-15T15:23:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.932+0000 7f7dd8600640 1 -- 172.21.15.150:0/980430821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 msgr2=0x7f7db8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.932+0000 7f7dd8600640 1 --2- 172.21.15.150:0/980430821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7db8004c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00c210 tx=0x7f7db8005830 comp rx=0 tx=0).stop 2024-04-15T15:23:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.932+0000 7f7dd6600640 1 -- 172.21.15.150:0/980430821 reap_dead start 2024-04-15T15:23:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.932+0000 7f7dd8600640 1 -- 172.21.15.150:0/980430821 shutdown_connections 2024-04-15T15:23:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.932+0000 7f7dd8600640 1 -- 172.21.15.150:0/980430821 >> 172.21.15.150:0/980430821 conn(0x7f7dd0073480 msgr2=0x7f7dd0073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.932+0000 7f7dd8600640 1 -- 172.21.15.150:0/980430821 shutdown_connections 2024-04-15T15:23:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.932+0000 7f7dd8600640 1 -- 172.21.15.150:0/980430821 wait complete. 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.934+0000 7f7dd8600640 1 Processor -- start 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.934+0000 7f7dd8600640 1 -- start start 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd8600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7dd0156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd01f5680 0x7f7dd01603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd8600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd0160920 0x7f7dd0160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd8600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00b8aa0 con 0x7f7db8004800 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd8600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00ba280 con 0x7f7dd0160920 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd00b9d00 con 0x7f7dd01f5680 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd5c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7dd0156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd5c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7dd0156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59210/0 (socket says 172.21.15.150:59210) 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd5c00640 1 -- 172.21.15.150:0/219920997 learned_addr learned my addr 172.21.15.150:0/219920997 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:51.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd5200640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd01f5680 0x7f7dd01603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd5c00640 1 -- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd01f5680 msgr2=0x7f7dd01603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd5c00640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd01f5680 0x7f7dd01603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd5c00640 1 -- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd0160920 msgr2=0x7f7dd0160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.935+0000 7f7dd5c00640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd0160920 0x7f7dd0160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.936+0000 7f7dd5c00640 1 -- 172.21.15.150:0/219920997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd00c8850 con 0x7f7db8004800 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.936+0000 7f7dd5c00640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7dd0156120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40108b0 tx=0x7f7dc4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.936+0000 7f7dcd600640 1 -- 172.21.15.150:0/219920997 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400f040 con 0x7f7db8004800 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.936+0000 7f7dcd600640 1 -- 172.21.15.150:0/219920997 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc400eb00 con 0x7f7db8004800 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.936+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd00c8b30 con 0x7f7db8004800 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.936+0000 7f7dcd600640 1 -- 172.21.15.150:0/219920997 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401c400 con 0x7f7db8004800 2024-04-15T15:23:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.936+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd00c90f0 con 0x7f7db8004800 2024-04-15T15:23:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.937+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd0068c80 con 0x7f7db8004800 2024-04-15T15:23:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.938+0000 7f7dcd600640 1 -- 172.21.15.150:0/219920997 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dc401c5a0 con 0x7f7db8004800 2024-04-15T15:23:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.939+0000 7f7dcd600640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7db006c920 0x7f7db006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.939+0000 7f7dcd600640 1 -- 172.21.15.150:0/219920997 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f7dc408bee0 con 0x7f7db8004800 2024-04-15T15:23:51.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.940+0000 7f7dd5200640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7db006c920 0x7f7db006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:51.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.940+0000 7f7dd5200640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7db006c920 0x7f7db006ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc034000 tx=0x7f7dbc009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:51.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:51.944+0000 7f7dcd600640 1 -- 172.21.15.150:0/219920997 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7dc4018030 con 0x7f7db8004800 2024-04-15T15:23:52.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.129+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7dd006bf20 con 0x7f7db8004800 2024-04-15T15:23:52.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.130+0000 7f7dcd600640 1 -- 172.21.15.150:0/219920997 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7f7dc405a5c0 con 0x7f7db8004800 2024-04-15T15:23:52.132 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7db006c920 msgr2=0x7f7db006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7db006c920 0x7f7db006ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc034000 tx=0x7f7dbc009030 comp rx=0 tx=0).stop 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 msgr2=0x7f7dd0156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7dd0156120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40108b0 tx=0x7f7dc4010d70 comp rx=0 tx=0).stop 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 shutdown_connections 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd0160920 0x7f7dd0160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7db006c920 0x7f7db006ede0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7dd01f5680 0x7f7dd01603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 --2- 172.21.15.150:0/219920997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db8004800 0x7f7dd0156120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 >> 172.21.15.150:0/219920997 conn(0x7f7dd0073480 msgr2=0x7f7dd00d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:52.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 shutdown_connections 2024-04-15T15:23:52.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.133+0000 7f7dd8600640 1 -- 172.21.15.150:0/219920997 wait complete. 2024-04-15T15:23:52.152 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-15T15:23:52.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.340+0000 7fa087200640 1 Processor -- start 2024-04-15T15:23:52.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.340+0000 7fa087200640 1 -- start start 2024-04-15T15:23:52.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa087200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 0x7fa0800cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa087200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa0800712a0 0x7fa08006b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa087200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0800cfe30 con 0x7fa08006be70 2024-04-15T15:23:52.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa087200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa0800d0110 con 0x7fa0800712a0 2024-04-15T15:23:52.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa087200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0800d03f0 con 0x7fa08015e8e0 2024-04-15T15:23:52.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa085400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa0800712a0 0x7fa08006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa085400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa0800712a0 0x7fa08006b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59222/0 (socket says 172.21.15.150:59222) 2024-04-15T15:23:52.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa085400640 1 -- 172.21.15.150:0/1612190052 learned_addr learned my addr 172.21.15.150:0/1612190052 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:52.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.342+0000 7fa085e00640 1 --2- 172.21.15.150:0/1612190052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 0x7fa0800cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.343+0000 7fa085400640 1 -- 172.21.15.150:0/1612190052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 msgr2=0x7fa0800cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:52.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.343+0000 7fa085400640 1 --2- 172.21.15.150:0/1612190052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 0x7fa0800cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.343+0000 7fa085400640 1 -- 172.21.15.150:0/1612190052 >> v1:172.21.15.187:6789/0 conn(0x7fa08006be70 legacy=0x7fa080072fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:52.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.343+0000 7fa085400640 1 -- 172.21.15.150:0/1612190052 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0800d06d0 con 0x7fa0800712a0 2024-04-15T15:23:52.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa085400640 1 --2- 172.21.15.150:0/1612190052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0800712a0 0x7fa08006b930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa070009870 tx=0x7fa07002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692c2bfb81481d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:52.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa084a00640 1 -- 172.21.15.150:0/1612190052 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070035020 con 0x7fa0800712a0 2024-04-15T15:23:52.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa084a00640 1 -- 172.21.15.150:0/1612190052 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa070031640 con 0x7fa0800712a0 2024-04-15T15:23:52.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa084a00640 1 -- 172.21.15.150:0/1612190052 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07002d040 con 0x7fa0800712a0 2024-04-15T15:23:52.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa087200640 1 -- 172.21.15.150:0/1612190052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0800712a0 msgr2=0x7fa08006b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa087200640 1 --2- 172.21.15.150:0/1612190052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0800712a0 0x7fa08006b930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa070009870 tx=0x7fa07002b8f0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa087200640 1 -- 172.21.15.150:0/1612190052 shutdown_connections 2024-04-15T15:23:52.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa087200640 1 --2- 172.21.15.150:0/1612190052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0800712a0 0x7fa08006b930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa087200640 1 --2- 172.21.15.150:0/1612190052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 0x7fa0800cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.344+0000 7fa087200640 1 -- 172.21.15.150:0/1612190052 >> 172.21.15.150:0/1612190052 conn(0x7fa0800b3370 msgr2=0x7fa0800b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.345+0000 7fa087200640 1 -- 172.21.15.150:0/1612190052 shutdown_connections 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.345+0000 7fa087200640 1 -- 172.21.15.150:0/1612190052 wait complete. 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.346+0000 7fa087200640 1 Processor -- start 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.346+0000 7fa087200640 1 -- start start 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa087200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08006be70 0x7fa080161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa087200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0800712a0 0x7fa080155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa087200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 0x7fa08014bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa087200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa0800b5be0 con 0x7fa08006be70 2024-04-15T15:23:52.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa087200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0800b5a60 con 0x7fa0800712a0 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa087200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0800b5d60 con 0x7fa08015e8e0 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa085e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08006be70 0x7fa080161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa085e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08006be70 0x7fa080161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59234/0 (socket says 172.21.15.150:59234) 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.347+0000 7fa085e00640 1 -- 172.21.15.150:0/1843009527 learned_addr learned my addr 172.21.15.150:0/1843009527 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa085e00640 1 -- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 msgr2=0x7fa08014bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa085e00640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 0x7fa08014bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa085e00640 1 -- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0800712a0 msgr2=0x7fa080155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa085e00640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0800712a0 0x7fa080155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa085e00640 1 -- 172.21.15.150:0/1843009527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa080158320 con 0x7fa08006be70 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa085e00640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08006be70 0x7fa080161810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa07400d620 tx=0x7fa07400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa07a000640 1 -- 172.21.15.150:0/1843009527 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07400b620 con 0x7fa08006be70 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa07a000640 1 -- 172.21.15.150:0/1843009527 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa074019070 con 0x7fa08006be70 2024-04-15T15:23:52.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa080158600 con 0x7fa08006be70 2024-04-15T15:23:52.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa07a000640 1 -- 172.21.15.150:0/1843009527 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07400ba00 con 0x7fa08006be70 2024-04-15T15:23:52.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.348+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa080156210 con 0x7fa08006be70 2024-04-15T15:23:52.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.349+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0800755c0 con 0x7fa08006be70 2024-04-15T15:23:52.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.350+0000 7fa07a000640 1 -- 172.21.15.150:0/1843009527 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa07400bba0 con 0x7fa08006be70 2024-04-15T15:23:52.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.351+0000 7fa07a000640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa06406c8f0 0x7fa06406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.351+0000 7fa07a000640 1 -- 172.21.15.150:0/1843009527 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fa07408bc40 con 0x7fa08006be70 2024-04-15T15:23:52.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.352+0000 7fa085400640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa06406c8f0 0x7fa06406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.353+0000 7fa085400640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa06406c8f0 0x7fa06406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa070036000 tx=0x7fa070030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:52.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.355+0000 7fa07a000640 1 -- 172.21.15.150:0/1843009527 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa07405a3b0 con 0x7fa08006be70 2024-04-15T15:23:52.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.542+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa08015bb20 con 0x7fa08006be70 2024-04-15T15:23:52.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.543+0000 7fa07a000640 1 -- 172.21.15.150:0/1843009527 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fa074059cb0 con 0x7fa08006be70 2024-04-15T15:23:52.545 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:52.545 INFO:teuthology.orchestra.run.smithi150.stdout:[ 2024-04-15T15:23:52.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:23:52.545 INFO:teuthology.orchestra.run.smithi150.stdout: "poolnum": 1, 2024-04-15T15:23:52.545 INFO:teuthology.orchestra.run.smithi150.stdout: "poolname": ".mgr" 2024-04-15T15:23:52.545 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:23:52.546 INFO:teuthology.orchestra.run.smithi150.stdout:] 2024-04-15T15:23:52.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.545+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa06406c8f0 msgr2=0x7fa06406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.545+0000 7fa087200640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa06406c8f0 0x7fa06406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa070036000 tx=0x7fa070030000 comp rx=0 tx=0).stop 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.545+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08006be70 msgr2=0x7fa080161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.545+0000 7fa087200640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08006be70 0x7fa080161810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa07400d620 tx=0x7fa07400dae0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.546+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 shutdown_connections 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.546+0000 7fa087200640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa06406c8f0 0x7fa06406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.546+0000 7fa087200640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa08015e8e0 0x7fa08014bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.546+0000 7fa087200640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0800712a0 0x7fa080155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.546+0000 7fa087200640 1 --2- 172.21.15.150:0/1843009527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08006be70 0x7fa080161810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.546+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 >> 172.21.15.150:0/1843009527 conn(0x7fa0800b3370 msgr2=0x7fa0800b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.546+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 shutdown_connections 2024-04-15T15:23:52.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.546+0000 7fa087200640 1 -- 172.21.15.150:0/1843009527 wait complete. 2024-04-15T15:23:52.565 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-15T15:23:52.565 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-15T15:23:52.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.780+0000 7f0567e00640 1 Processor -- start 2024-04-15T15:23:52.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.781+0000 7f0567e00640 1 -- start start 2024-04-15T15:23:52.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.782+0000 7f0567e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0568071f10 0x7f05680ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.782+0000 7f0567e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f056806f230 0x7f056806f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.782+0000 7f0567e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05680b37f0 con 0x7f05680cf2c0 2024-04-15T15:23:52.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.782+0000 7f0567e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f056806eea0 con 0x7f056806f230 2024-04-15T15:23:52.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.782+0000 7f0567e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f05681630e0 con 0x7f0568071f10 2024-04-15T15:23:52.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.782+0000 7f0566a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f05680cf2c0 0x7f056806de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45872/0 (socket says 172.21.15.150:45872) 2024-04-15T15:23:52.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.782+0000 7f0566a00640 1 -- 172.21.15.150:0/3169847714 learned_addr learned my addr 172.21.15.150:0/3169847714 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0566000640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0568071f10 0x7f05680ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0567400640 1 --2- 172.21.15.150:0/3169847714 >> v2:172.21.15.187:3300/0 conn(0x7f056806f230 0x7f056806f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194825530 0 0) 0x7f05680b37f0 con 0x7f05680cf2c0 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f054c003520 con 0x7f05680cf2c0 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 779254833 0 0) 0x7f054c003520 con 0x7f05680cf2c0 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0568071f10 msgr2=0x7f05680ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0568071f10 0x7f05680ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 >> v2:172.21.15.187:3300/0 conn(0x7f056806f230 msgr2=0x7f056806f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 --2- 172.21.15.150:0/3169847714 >> v2:172.21.15.187:3300/0 conn(0x7f056806f230 0x7f056806f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056806e600 con 0x7f05680cf2c0 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f05580019f0 con 0x7f05680cf2c0 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 >> v1:172.21.15.150:6789/0 conn(0x7f05680cf2c0 legacy=0x7f056806de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0568071f10 0x7f054c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f054c003f30 0x7f054c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f054c004d40 0x7f054c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f056806eea0 con 0x7f054c004d40 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f05681630e0 con 0x7f054c003f30 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.783+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f054c00b9c0 con 0x7f0568071f10 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.784+0000 7f0567400640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f054c004d40 0x7f054c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.784+0000 7f0567400640 1 -- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0568071f10 msgr2=0x7f054c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.784+0000 7f0567400640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0568071f10 0x7f054c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.784+0000 7f0567400640 1 -- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f054c003f30 msgr2=0x7f054c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:52.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.784+0000 7f0567400640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f054c003f30 0x7f054c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.784+0000 7f0567400640 1 -- 172.21.15.150:0/3169847714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c003520 con 0x7f054c004d40 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.785+0000 7f0567400640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f054c004d40 0x7f054c0051c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0560009ac0 tx=0x7f054c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88701f19d055615d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.785+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560039070 con 0x7f054c004d40 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.786+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056002c6d0 con 0x7f054c004d40 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.786+0000 7f0565600640 1 -- 172.21.15.150:0/3169847714 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560009690 con 0x7f054c004d40 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.786+0000 7f0567e00640 1 -- 172.21.15.150:0/3169847714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f054c004d40 msgr2=0x7f054c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.786+0000 7f0567e00640 1 --2- 172.21.15.150:0/3169847714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f054c004d40 0x7f054c0051c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0560009ac0 tx=0x7f054c005830 comp rx=0 tx=0).stop 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.786+0000 7f0567400640 1 -- 172.21.15.150:0/3169847714 reap_dead start 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.786+0000 7f0567e00640 1 -- 172.21.15.150:0/3169847714 shutdown_connections 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.786+0000 7f0567e00640 1 -- 172.21.15.150:0/3169847714 >> 172.21.15.150:0/3169847714 conn(0x7f05680b6170 msgr2=0x7f056806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.786+0000 7f0567e00640 1 -- 172.21.15.150:0/3169847714 shutdown_connections 2024-04-15T15:23:52.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.787+0000 7f0567e00640 1 -- 172.21.15.150:0/3169847714 wait complete. 2024-04-15T15:23:52.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.788+0000 7f0567e00640 1 Processor -- start 2024-04-15T15:23:52.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.788+0000 7f0567e00640 1 -- start start 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.789+0000 7f0567e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f054c004d40 0x7f0568143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.789+0000 7f0567e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0568144340 0x7f05681fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.789+0000 7f0567e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05681fc180 0x7f056815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.789+0000 7f0567e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f05680cf6a0 con 0x7f0568144340 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.789+0000 7f0567e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f056806ebc0 con 0x7f05681fc180 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.789+0000 7f0567e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f05680cf820 con 0x7f054c004d40 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.789+0000 7f0566000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0568144340 0x7f05681fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0568144340 0x7f05681fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59246/0 (socket says 172.21.15.150:59246) 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566000640 1 -- 172.21.15.150:0/2579808538 learned_addr learned my addr 172.21.15.150:0/2579808538 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:52.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566a00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f054c004d40 0x7f0568143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566000640 1 -- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f054c004d40 msgr2=0x7f0568143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566000640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f054c004d40 0x7f0568143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566000640 1 -- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05681fc180 msgr2=0x7f056815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566000640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05681fc180 0x7f056815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566000640 1 -- 172.21.15.150:0/2579808538 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056815b230 con 0x7f0568144340 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0566000640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0568144340 0x7f05681fbc40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f055000e7b0 tx=0x7f055000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f055f400640 1 -- 172.21.15.150:0/2579808538 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550017070 con 0x7f0568144340 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f055f400640 1 -- 172.21.15.150:0/2579808538 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055000c860 con 0x7f0568144340 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f055f400640 1 -- 172.21.15.150:0/2579808538 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055000cc50 con 0x7f0568144340 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.790+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0568155250 con 0x7f0568144340 2024-04-15T15:23:52.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.791+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0568155810 con 0x7f0568144340 2024-04-15T15:23:52.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.792+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05681fc5a0 con 0x7f0568144340 2024-04-15T15:23:52.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.793+0000 7f055f400640 1 -- 172.21.15.150:0/2579808538 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f055001a070 con 0x7f0568144340 2024-04-15T15:23:52.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.794+0000 7f055f400640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:52.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.794+0000 7f055f400640 1 -- 172.21.15.150:0/2579808538 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f055008b970 con 0x7f0568144340 2024-04-15T15:23:52.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.795+0000 7f0566a00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:52.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.796+0000 7f0566a00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f054406c8f0 0x7f054406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f05580012e0 tx=0x7f0558009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:52.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.800+0000 7f055f400640 1 -- 172.21.15.150:0/2579808538 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f056815b230 con 0x7f0568144340 2024-04-15T15:23:52.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:52.995+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0568068b40 con 0x7f0568144340 2024-04-15T15:23:53.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.193+0000 7f055f400640 1 -- 172.21.15.150:0/2579808538 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f055005a0e0 con 0x7f0568144340 2024-04-15T15:23:53.195 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_metadata' created 2024-04-15T15:23:53.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.196+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f054406c8f0 msgr2=0x7f054406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:53.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.196+0000 7f0567e00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f054406c8f0 0x7f054406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f05580012e0 tx=0x7f0558009040 comp rx=0 tx=0).stop 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.196+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0568144340 msgr2=0x7f05681fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.196+0000 7f0567e00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0568144340 0x7f05681fbc40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f055000e7b0 tx=0x7f055000ec70 comp rx=0 tx=0).stop 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.197+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 shutdown_connections 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.197+0000 7f0567e00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05681fc180 0x7f056815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.197+0000 7f0567e00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0568144340 0x7f05681fbc40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.197+0000 7f0567e00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.197+0000 7f0567e00640 1 --2- 172.21.15.150:0/2579808538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f054c004d40 0x7f0568143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.197+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 >> 172.21.15.150:0/2579808538 conn(0x7f05680b6170 msgr2=0x7f0568075180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:53.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.197+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 shutdown_connections 2024-04-15T15:23:53.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.197+0000 7f0567e00640 1 -- 172.21.15.150:0/2579808538 wait complete. 2024-04-15T15:23:53.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-15T15:23:53.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.422+0000 7f3d37400640 1 Processor -- start 2024-04-15T15:23:53.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.422+0000 7f3d37400640 1 -- start start 2024-04-15T15:23:53.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.423+0000 7f3d37400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d3006fc80 0x7f3d3015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.423+0000 7f3d37400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3d30072150 0x7f3d300725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.423+0000 7f3d37400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d3015d520 con 0x7f3d3015a440 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.423+0000 7f3d37400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3d30072b00 con 0x7f3d30072150 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.423+0000 7f3d37400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3006e280 con 0x7f3d3006fc80 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.424+0000 7f3d34a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3d3015a440 0x7f3d300cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45882/0 (socket says 172.21.15.150:45882) 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.424+0000 7f3d34a00640 1 -- 172.21.15.150:0/2885011111 learned_addr learned my addr 172.21.15.150:0/2885011111 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.424+0000 7f3d35400640 1 --2- 172.21.15.150:0/2885011111 >> v2:172.21.15.187:3300/0 conn(0x7f3d30072150 0x7f3d300725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.424+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654311559 0 0) 0x7f3d3015d520 con 0x7f3d3015a440 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.424+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d18003520 con 0x7f3d3015a440 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.424+0000 7f3d2fe00640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d3006fc80 0x7f3d3015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1418073808 0 0) 0x7f3d18003520 con 0x7f3d3015a440 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d3006fc80 msgr2=0x7f3d3015cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d3006fc80 0x7f3d3015cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 >> v2:172.21.15.187:3300/0 conn(0x7f3d30072150 msgr2=0x7f3d300725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 --2- 172.21.15.150:0/2885011111 >> v2:172.21.15.187:3300/0 conn(0x7f3d30072150 0x7f3d300725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3015d800 con 0x7f3d3015a440 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f3d24001690 con 0x7f3d3015a440 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 >> v1:172.21.15.150:6789/0 conn(0x7f3d3015a440 legacy=0x7f3d300cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d3006fc80 0x7f3d180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18003f30 0x7f3d18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d18004d40 0x7f3d180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30072b00 con 0x7f3d18004d40 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3006e280 con 0x7f3d18003f30 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1800ba10 con 0x7f3d3006fc80 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d35400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d18004d40 0x7f3d180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d34a00640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d3006fc80 0x7f3d180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d2fe00640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18003f30 0x7f3d18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d35400640 1 -- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d3006fc80 msgr2=0x7f3d180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:53.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d35400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d3006fc80 0x7f3d180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d35400640 1 -- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18003f30 msgr2=0x7f3d18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:53.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d35400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18003f30 0x7f3d18004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.425+0000 7f3d35400640 1 -- 172.21.15.150:0/2885011111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d18003520 con 0x7f3d18004d40 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.426+0000 7f3d35400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d18004d40 0x7f3d180051c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009fc0 tx=0x7f3d18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18b99f622271c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28039070 con 0x7f3d18004d40 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2802cc60 con 0x7f3d18004d40 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d2f400640 1 -- 172.21.15.150:0/2885011111 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28009c10 con 0x7f3d18004d40 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d37400640 1 -- 172.21.15.150:0/2885011111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d18004d40 msgr2=0x7f3d180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d37400640 1 --2- 172.21.15.150:0/2885011111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d18004d40 0x7f3d180051c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009fc0 tx=0x7f3d18005830 comp rx=0 tx=0).stop 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d35400640 1 -- 172.21.15.150:0/2885011111 reap_dead start 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d37400640 1 -- 172.21.15.150:0/2885011111 shutdown_connections 2024-04-15T15:23:53.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d37400640 1 -- 172.21.15.150:0/2885011111 >> 172.21.15.150:0/2885011111 conn(0x7f3d300b2460 msgr2=0x7f3d300b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d37400640 1 -- 172.21.15.150:0/2885011111 shutdown_connections 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.427+0000 7f3d37400640 1 -- 172.21.15.150:0/2885011111 wait complete. 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.429+0000 7f3d37400640 1 Processor -- start 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.429+0000 7f3d37400640 1 -- start start 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d37400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18004d40 0x7f3d30144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d37400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d300c5bc0 0x7f3d300c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d37400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d300aed20 0x7f3d300af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d37400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3015a820 con 0x7f3d300aed20 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d37400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d300cd7f0 con 0x7f3d18004d40 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d37400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3015ddc0 con 0x7f3d300c5bc0 2024-04-15T15:23:53.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d35400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d300aed20 0x7f3d300af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d35400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d300aed20 0x7f3d300af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59278/0 (socket says 172.21.15.150:59278) 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.430+0000 7f3d35400640 1 -- 172.21.15.150:0/3333313651 learned_addr learned my addr 172.21.15.150:0/3333313651 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d34a00640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18004d40 0x7f3d30144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d2fe00640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d300c5bc0 0x7f3d300c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d35400640 1 -- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d300c5bc0 msgr2=0x7f3d300c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d35400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d300c5bc0 0x7f3d300c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d35400640 1 -- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18004d40 msgr2=0x7f3d30144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d35400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18004d40 0x7f3d30144940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:53.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d35400640 1 -- 172.21.15.150:0/3333313651 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d300c9cd0 con 0x7f3d300aed20 2024-04-15T15:23:53.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d35400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d300aed20 0x7f3d300af1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e230 tx=0x7f3d28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:53.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d2cc00640 1 -- 172.21.15.150:0/3333313651 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28042020 con 0x7f3d300aed20 2024-04-15T15:23:53.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d2cc00640 1 -- 172.21.15.150:0/3333313651 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2803a040 con 0x7f3d300aed20 2024-04-15T15:23:53.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d2cc00640 1 -- 172.21.15.150:0/3333313651 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28039030 con 0x7f3d300aed20 2024-04-15T15:23:53.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d300c7cc0 con 0x7f3d300aed20 2024-04-15T15:23:53.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.431+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d300c8280 con 0x7f3d300aed20 2024-04-15T15:23:53.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.432+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d300af6e0 con 0x7f3d300aed20 2024-04-15T15:23:53.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.433+0000 7f3d2cc00640 1 -- 172.21.15.150:0/3333313651 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d28005d10 con 0x7f3d300aed20 2024-04-15T15:23:53.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.435+0000 7f3d2cc00640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3d1006c920 0x7f3d1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:53.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.435+0000 7f3d2cc00640 1 -- 172.21.15.150:0/3333313651 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f3d280ad9e0 con 0x7f3d300aed20 2024-04-15T15:23:53.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.435+0000 7f3d34a00640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3d1006c920 0x7f3d1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:53.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.437+0000 7f3d34a00640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3d1006c920 0x7f3d1006ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d300b54f0 tx=0x7f3d24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:53.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.440+0000 7f3d2cc00640 1 -- 172.21.15.150:0/3333313651 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d2804c020 con 0x7f3d300aed20 2024-04-15T15:23:53.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:53.642+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3d30068c80 con 0x7f3d300aed20 2024-04-15T15:23:54.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.198+0000 7f3d2cc00640 1 -- 172.21.15.150:0/3333313651 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f3d28080e20 con 0x7f3d300aed20 2024-04-15T15:23:54.200 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data' created 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3d1006c920 msgr2=0x7f3d1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3d1006c920 0x7f3d1006ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d300b54f0 tx=0x7f3d24009040 comp rx=0 tx=0).stop 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d300aed20 msgr2=0x7f3d300af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d300aed20 0x7f3d300af1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e230 tx=0x7f3d28009750 comp rx=0 tx=0).stop 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 shutdown_connections 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d300aed20 0x7f3d300af1a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3d1006c920 0x7f3d1006ede0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d300c5bc0 0x7f3d300c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 --2- 172.21.15.150:0/3333313651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d18004d40 0x7f3d30144940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 >> 172.21.15.150:0/3333313651 conn(0x7f3d300b2460 msgr2=0x7f3d300cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:54.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 shutdown_connections 2024-04-15T15:23:54.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.202+0000 7f3d37400640 1 -- 172.21.15.150:0/3333313651 wait complete. 2024-04-15T15:23:54.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-15T15:23:54.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.419+0000 7fd168a00640 1 Processor -- start 2024-04-15T15:23:54.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.419+0000 7fd168a00640 1 -- start start 2024-04-15T15:23:54.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.420+0000 7fd168a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd16415e9a0 0x7fd164162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:54.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd168a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd1640cfc00 0x7fd1640d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:54.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd168a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd164163450 con 0x7fd1640ba3e0 2024-04-15T15:23:54.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd168a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fd164162e90 con 0x7fd16415e9a0 2024-04-15T15:23:54.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd168a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd164163170 con 0x7fd1640cfc00 2024-04-15T15:23:54.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd163400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd16415e9a0 0x7fd164162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:54.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd162a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd1640ba3e0 0x7fd1640cf4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45890/0 (socket says 172.21.15.150:45890) 2024-04-15T15:23:54.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd162a00640 1 -- 172.21.15.150:0/1422746793 learned_addr learned my addr 172.21.15.150:0/1422746793 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:54.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd163e00640 1 --2- 172.21.15.150:0/1422746793 >> v2:172.21.15.187:3300/0 conn(0x7fd1640cfc00 0x7fd1640d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:54.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd163400640 1 -- 172.21.15.150:0/1422746793 >> v2:172.21.15.187:3300/0 conn(0x7fd1640cfc00 msgr2=0x7fd1640d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:54.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd163400640 1 --2- 172.21.15.150:0/1422746793 >> v2:172.21.15.187:3300/0 conn(0x7fd1640cfc00 0x7fd1640d3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd163400640 1 -- 172.21.15.150:0/1422746793 >> v1:172.21.15.150:6789/0 conn(0x7fd1640ba3e0 legacy=0x7fd1640cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:54.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd163400640 1 -- 172.21.15.150:0/1422746793 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1640d2920 con 0x7fd16415e9a0 2024-04-15T15:23:54.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.421+0000 7fd162000640 1 -- 172.21.15.150:0/1422746793 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2831888780 0 0) 0x7fd164163450 con 0x7fd1640ba3e0 2024-04-15T15:23:54.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.423+0000 7fd163400640 1 --2- 172.21.15.150:0/1422746793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd16415e9a0 0x7fd164162590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd16408ce80 tx=0x7fd15402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ead7924e355fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:54.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.423+0000 7fd162000640 1 -- 172.21.15.150:0/1422746793 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154039070 con 0x7fd16415e9a0 2024-04-15T15:23:54.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.423+0000 7fd162000640 1 -- 172.21.15.150:0/1422746793 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15402cd00 con 0x7fd16415e9a0 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.423+0000 7fd162000640 1 -- 172.21.15.150:0/1422746793 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154034660 con 0x7fd16415e9a0 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.423+0000 7fd168a00640 1 -- 172.21.15.150:0/1422746793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd16415e9a0 msgr2=0x7fd164162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.423+0000 7fd168a00640 1 --2- 172.21.15.150:0/1422746793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd16415e9a0 0x7fd164162590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd16408ce80 tx=0x7fd15402bf50 comp rx=0 tx=0).stop 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.424+0000 7fd168a00640 1 -- 172.21.15.150:0/1422746793 shutdown_connections 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.424+0000 7fd168a00640 1 --2- 172.21.15.150:0/1422746793 >> v2:172.21.15.187:3300/0 conn(0x7fd1640cfc00 0x7fd1640d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.424+0000 7fd168a00640 1 --2- 172.21.15.150:0/1422746793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd16415e9a0 0x7fd164162590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.424+0000 7fd168a00640 1 -- 172.21.15.150:0/1422746793 >> 172.21.15.150:0/1422746793 conn(0x7fd16406ce40 msgr2=0x7fd16406d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.424+0000 7fd168a00640 1 -- 172.21.15.150:0/1422746793 shutdown_connections 2024-04-15T15:23:54.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.424+0000 7fd168a00640 1 -- 172.21.15.150:0/1422746793 wait complete. 2024-04-15T15:23:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.425+0000 7fd168a00640 1 Processor -- start 2024-04-15T15:23:54.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.426+0000 7fd168a00640 1 -- start start 2024-04-15T15:23:54.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.426+0000 7fd168a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640ba3e0 0x7fd16414ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:54.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.426+0000 7fd168a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1640cfc00 0x7fd1640c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:54.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.426+0000 7fd168a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd16415e9a0 0x7fd16415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.426+0000 7fd168a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd16415ed80 con 0x7fd1640cfc00 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.426+0000 7fd168a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd164162ad0 con 0x7fd16415e9a0 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.426+0000 7fd168a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1640c8790 con 0x7fd1640ba3e0 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd162a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1640cfc00 0x7fd1640c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd162a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1640cfc00 0x7fd1640c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59292/0 (socket says 172.21.15.150:59292) 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd162a00640 1 -- 172.21.15.150:0/1272674870 learned_addr learned my addr 172.21.15.150:0/1272674870 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd163400640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640ba3e0 0x7fd16414ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd163e00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd16415e9a0 0x7fd16415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd162a00640 1 -- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640ba3e0 msgr2=0x7fd16414ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd162a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640ba3e0 0x7fd16414ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd162a00640 1 -- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd16415e9a0 msgr2=0x7fd16415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:54.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd162a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd16415e9a0 0x7fd16415fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd162a00640 1 -- 172.21.15.150:0/1272674870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd164161c80 con 0x7fd1640cfc00 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.427+0000 7fd163400640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640ba3e0 0x7fd16414ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.428+0000 7fd162a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1640cfc00 0x7fd1640c8250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00c500 tx=0x7fd14c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.428+0000 7fd15b400640 1 -- 172.21.15.150:0/1272674870 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c018070 con 0x7fd1640cfc00 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.428+0000 7fd15b400640 1 -- 172.21.15.150:0/1272674870 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14c007580 con 0x7fd1640cfc00 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.428+0000 7fd15b400640 1 -- 172.21.15.150:0/1272674870 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c004a20 con 0x7fd1640cfc00 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.428+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1641509e0 con 0x7fd1640cfc00 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.428+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd164161950 con 0x7fd1640cfc00 2024-04-15T15:23:54.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.429+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1640755c0 con 0x7fd1640cfc00 2024-04-15T15:23:54.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.430+0000 7fd15b400640 1 -- 172.21.15.150:0/1272674870 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd14c004bc0 con 0x7fd1640cfc00 2024-04-15T15:23:54.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.431+0000 7fd15b400640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd14006c8f0 0x7fd14006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:54.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.431+0000 7fd15b400640 1 -- 172.21.15.150:0/1272674870 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7fd14c08ad90 con 0x7fd1640cfc00 2024-04-15T15:23:54.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.432+0000 7fd163400640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd14006c8f0 0x7fd14006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:54.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.433+0000 7fd163400640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd14006c8f0 0x7fd14006edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd154036000 tx=0x7fd154030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:54.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.437+0000 7fd15b400640 1 -- 172.21.15.150:0/1272674870 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd14c059210 con 0x7fd1640cfc00 2024-04-15T15:23:54.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:54.681+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd164068ef0 con 0x7fd1640cfc00 2024-04-15T15:23:55.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.262+0000 7fd15b400640 1 -- 172.21.15.150:0/1272674870 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fd14c058b10 con 0x7fd1640cfc00 2024-04-15T15:23:55.264 INFO:teuthology.orchestra.run.smithi150.stderr:new fs with metadata pool 2 and data pool 3 2024-04-15T15:23:55.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.268+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd14006c8f0 msgr2=0x7fd14006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.268+0000 7fd168a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd14006c8f0 0x7fd14006edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd154036000 tx=0x7fd154030000 comp rx=0 tx=0).stop 2024-04-15T15:23:55.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.268+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1640cfc00 msgr2=0x7fd1640c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.268+0000 7fd168a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1640cfc00 0x7fd1640c8250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00c500 tx=0x7fd14c00c9c0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.269+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 shutdown_connections 2024-04-15T15:23:55.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.269+0000 7fd168a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd16415e9a0 0x7fd16415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.269+0000 7fd168a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1640cfc00 0x7fd1640c8250 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.269+0000 7fd168a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd14006c8f0 0x7fd14006edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.269+0000 7fd168a00640 1 --2- 172.21.15.150:0/1272674870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640ba3e0 0x7fd16414ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.269+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 >> 172.21.15.150:0/1272674870 conn(0x7fd16406ce40 msgr2=0x7fd164153720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:55.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.269+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 shutdown_connections 2024-04-15T15:23:55.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.269+0000 7fd168a00640 1 -- 172.21.15.150:0/1272674870 wait complete. 2024-04-15T15:23:55.298 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:55.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.476+0000 7f5860800640 1 Processor -- start 2024-04-15T15:23:55.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.476+0000 7f5860800640 1 -- start start 2024-04-15T15:23:55.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f5860800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f58580cf8c0 0x7f5858169770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f5860800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58580b89f0 con 0x7f58580d3e20 2024-04-15T15:23:55.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f5860800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58580cfdc0 con 0x7f585815ee00 2024-04-15T15:23:55.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f5860800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f58580d2b10 con 0x7f58580cf8c0 2024-04-15T15:23:55.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f585de00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f58580d3e20 0x7f58580d4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45892/0 (socket says 172.21.15.150:45892) 2024-04-15T15:23:55.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f585de00640 1 -- 172.21.15.150:0/3748279840 learned_addr learned my addr 172.21.15.150:0/3748279840 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:55.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f585e800640 1 --2- 172.21.15.150:0/3748279840 >> v2:172.21.15.150:3300/0 conn(0x7f58580cf8c0 0x7f5858169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f585ca00640 1 -- 172.21.15.150:0/3748279840 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188228071 0 0) 0x7f58580b89f0 con 0x7f58580d3e20 2024-04-15T15:23:55.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.478+0000 7f585ca00640 1 -- 172.21.15.150:0/3748279840 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5840003520 con 0x7f58580d3e20 2024-04-15T15:23:55.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.479+0000 7f585e800640 1 -- 172.21.15.150:0/3748279840 >> v1:172.21.15.187:6789/0 conn(0x7f585815ee00 legacy=0x7f58580cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:55.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.479+0000 7f585e800640 1 -- 172.21.15.150:0/3748279840 >> v1:172.21.15.150:6789/0 conn(0x7f58580d3e20 legacy=0x7f58580d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.479+0000 7f585e800640 1 -- 172.21.15.150:0/3748279840 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58580d2d60 con 0x7f58580cf8c0 2024-04-15T15:23:55.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.480+0000 7f585e800640 1 --2- 172.21.15.150:0/3748279840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580cf8c0 0x7f5858169770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5854009d90 tx=0x7f585402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4231ec72ae7422 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:55.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.480+0000 7f585ca00640 1 -- 172.21.15.150:0/3748279840 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854039070 con 0x7f58580cf8c0 2024-04-15T15:23:55.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.480+0000 7f585ca00640 1 -- 172.21.15.150:0/3748279840 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585402cd00 con 0x7f58580cf8c0 2024-04-15T15:23:55.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.480+0000 7f585ca00640 1 -- 172.21.15.150:0/3748279840 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854034660 con 0x7f58580cf8c0 2024-04-15T15:23:55.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.480+0000 7f5860800640 1 -- 172.21.15.150:0/3748279840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580cf8c0 msgr2=0x7f5858169770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.480+0000 7f5860800640 1 --2- 172.21.15.150:0/3748279840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580cf8c0 0x7f5858169770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5854009d90 tx=0x7f585402bf50 comp rx=0 tx=0).stop 2024-04-15T15:23:55.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.481+0000 7f5860800640 1 -- 172.21.15.150:0/3748279840 shutdown_connections 2024-04-15T15:23:55.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.481+0000 7f5860800640 1 --2- 172.21.15.150:0/3748279840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580cf8c0 0x7f5858169770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.481+0000 7f5860800640 1 -- 172.21.15.150:0/3748279840 >> 172.21.15.150:0/3748279840 conn(0x7f585806d470 msgr2=0x7f585806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:55.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.481+0000 7f5860800640 1 -- 172.21.15.150:0/3748279840 shutdown_connections 2024-04-15T15:23:55.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.481+0000 7f5860800640 1 -- 172.21.15.150:0/3748279840 wait complete. 2024-04-15T15:23:55.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.482+0000 7f5860800640 1 Processor -- start 2024-04-15T15:23:55.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.482+0000 7f5860800640 1 -- start start 2024-04-15T15:23:55.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.483+0000 7f5860800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58580cf8c0 0x7f5858158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.483+0000 7f5860800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580d3e20 0x7f58580ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.483+0000 7f5860800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585815ee00 0x7f58580cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.483+0000 7f5860800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f58580d3840 con 0x7f58580d3e20 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.483+0000 7f5860800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58580d33b0 con 0x7f585815ee00 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.483+0000 7f5860800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f58580d39c0 con 0x7f58580cf8c0 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580d3e20 0x7f58580ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580d3e20 0x7f58580ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59308/0 (socket says 172.21.15.150:59308) 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 -- 172.21.15.150:0/4012979178 learned_addr learned my addr 172.21.15.150:0/4012979178 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585e800640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585815ee00 0x7f58580cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585de00640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58580cf8c0 0x7f5858158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 -- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58580cf8c0 msgr2=0x7f5858158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58580cf8c0 0x7f5858158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 -- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585815ee00 msgr2=0x7f58580cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585815ee00 0x7f58580cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 -- 172.21.15.150:0/4012979178 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58580b41a0 con 0x7f58580d3e20 2024-04-15T15:23:55.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585de00640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58580cf8c0 0x7f5858158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:55.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f585d400640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580d3e20 0x7f58580ca850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f584800b3c0 tx=0x7f584800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:55.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.484+0000 7f5852000640 1 -- 172.21.15.150:0/4012979178 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848011830 con 0x7f58580d3e20 2024-04-15T15:23:55.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.485+0000 7f5852000640 1 -- 172.21.15.150:0/4012979178 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5848016070 con 0x7f58580d3e20 2024-04-15T15:23:55.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.485+0000 7f5852000640 1 -- 172.21.15.150:0/4012979178 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848011c50 con 0x7f58580d3e20 2024-04-15T15:23:55.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.485+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58580b4480 con 0x7f58580d3e20 2024-04-15T15:23:55.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.485+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58580b4a40 con 0x7f58580d3e20 2024-04-15T15:23:55.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.486+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5858068c80 con 0x7f58580d3e20 2024-04-15T15:23:55.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.487+0000 7f5852000640 1 -- 172.21.15.150:0/4012979178 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f584801b460 con 0x7f58580d3e20 2024-04-15T15:23:55.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.488+0000 7f5852000640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f583c06c8f0 0x7f583c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.488+0000 7f5852000640 1 -- 172.21.15.150:0/4012979178 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f584808b680 con 0x7f58580d3e20 2024-04-15T15:23:55.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.488+0000 7f585de00640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f583c06c8f0 0x7f583c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.489+0000 7f585de00640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f583c06c8f0 0x7f583c06edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f584c005880 tx=0x7f584c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:55.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.493+0000 7f5852000640 1 -- 172.21.15.150:0/4012979178 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5848020020 con 0x7f58580d3e20 2024-04-15T15:23:55.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.694+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58580cf100 con 0x7f58580d3e20 2024-04-15T15:23:55.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.697+0000 7f5852000640 1 -- 172.21.15.150:0/4012979178 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16666 (secure 0 0 0) 0x7f5848059a10 con 0x7f58580d3e20 2024-04-15T15:23:55.699 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:55.699 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":21,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:54.683131+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:55.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f583c06c8f0 msgr2=0x7f583c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f583c06c8f0 0x7f583c06edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f584c005880 tx=0x7f584c009040 comp rx=0 tx=0).stop 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580d3e20 msgr2=0x7f58580ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580d3e20 0x7f58580ca850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f584800b3c0 tx=0x7f584800b880 comp rx=0 tx=0).stop 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 shutdown_connections 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585815ee00 0x7f58580cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58580d3e20 0x7f58580ca850 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f583c06c8f0 0x7f583c06edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 --2- 172.21.15.150:0/4012979178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58580cf8c0 0x7f5858158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 >> 172.21.15.150:0/4012979178 conn(0x7f585806d470 msgr2=0x7f58580d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 shutdown_connections 2024-04-15T15:23:55.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.700+0000 7f5860800640 1 -- 172.21.15.150:0/4012979178 wait complete. 2024-04-15T15:23:55.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:55.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.920+0000 7fd4fa000640 1 Processor -- start 2024-04-15T15:23:55.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.920+0000 7fd4fa000640 1 -- start start 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.921+0000 7fd4fa000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd4f40cefc0 0x7fd4f40b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.921+0000 7fd4fa000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f40b3e10 0x7fd4f415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.921+0000 7fd4fa000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4f40b4be0 con 0x7fd4f41461f0 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.921+0000 7fd4fa000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f40a9640 con 0x7fd4f40cefc0 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.921+0000 7fd4fa000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f40b4620 con 0x7fd4f40b3e10 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4f3400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd4f41461f0 0x7fd4f40ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45898/0 (socket says 172.21.15.150:45898) 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4f3400640 1 -- 172.21.15.150:0/889832444 learned_addr learned my addr 172.21.15.150:0/889832444 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4eaa00640 1 --2- 172.21.15.150:0/889832444 >> v2:172.21.15.150:3300/0 conn(0x7fd4f40cefc0 0x7fd4f40b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4f3e00640 1 --2- 172.21.15.150:0/889832444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f40b3e10 0x7fd4f415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4f2a00640 1 -- 172.21.15.150:0/889832444 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1906458507 0 0) 0x7fd4f40b4be0 con 0x7fd4f41461f0 2024-04-15T15:23:55.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4f2a00640 1 -- 172.21.15.150:0/889832444 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4d8003520 con 0x7fd4f41461f0 2024-04-15T15:23:55.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4eaa00640 1 -- 172.21.15.150:0/889832444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f40b3e10 msgr2=0x7fd4f415d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:55.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4eaa00640 1 --2- 172.21.15.150:0/889832444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f40b3e10 0x7fd4f415d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4eaa00640 1 -- 172.21.15.150:0/889832444 >> v1:172.21.15.150:6789/0 conn(0x7fd4f41461f0 legacy=0x7fd4f40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.922+0000 7fd4eaa00640 1 -- 172.21.15.150:0/889832444 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f40b4900 con 0x7fd4f40cefc0 2024-04-15T15:23:55.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.923+0000 7fd4eaa00640 1 --2- 172.21.15.150:0/889832444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40cefc0 0x7fd4f40b38d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0097a0 tx=0x7fd4dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f5f742b4a234a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:55.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4f2a00640 1 -- 172.21.15.150:0/889832444 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc035020 con 0x7fd4f40cefc0 2024-04-15T15:23:55.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4f2a00640 1 -- 172.21.15.150:0/889832444 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4dc031830 con 0x7fd4f40cefc0 2024-04-15T15:23:55.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4f2a00640 1 -- 172.21.15.150:0/889832444 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc031c00 con 0x7fd4f40cefc0 2024-04-15T15:23:55.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4fa000640 1 -- 172.21.15.150:0/889832444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40cefc0 msgr2=0x7fd4f40b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4fa000640 1 --2- 172.21.15.150:0/889832444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40cefc0 0x7fd4f40b38d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0097a0 tx=0x7fd4dc02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4fa000640 1 -- 172.21.15.150:0/889832444 shutdown_connections 2024-04-15T15:23:55.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4fa000640 1 --2- 172.21.15.150:0/889832444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f40b3e10 0x7fd4f415d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4fa000640 1 --2- 172.21.15.150:0/889832444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40cefc0 0x7fd4f40b38d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4fa000640 1 -- 172.21.15.150:0/889832444 >> 172.21.15.150:0/889832444 conn(0x7fd4f40b27b0 msgr2=0x7fd4f40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:55.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4fa000640 1 -- 172.21.15.150:0/889832444 shutdown_connections 2024-04-15T15:23:55.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.924+0000 7fd4fa000640 1 -- 172.21.15.150:0/889832444 wait complete. 2024-04-15T15:23:55.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.926+0000 7fd4fa000640 1 Processor -- start 2024-04-15T15:23:55.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.926+0000 7fd4fa000640 1 -- start start 2024-04-15T15:23:55.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.927+0000 7fd4fa000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40b3e10 0x7fd4f41e7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.927+0000 7fd4fa000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4f40cefc0 0x7fd4f41e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.927+0000 7fd4fa000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f41461f0 0x7fd4f415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.927+0000 7fd4fa000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41465d0 con 0x7fd4f40b3e10 2024-04-15T15:23:55.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.927+0000 7fd4fa000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f415d290 con 0x7fd4f40cefc0 2024-04-15T15:23:55.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.927+0000 7fd4fa000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f40b4280 con 0x7fd4f41461f0 2024-04-15T15:23:55.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.927+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40b3e10 0x7fd4f41e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40b3e10 0x7fd4f41e7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59322/0 (socket says 172.21.15.150:59322) 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3400640 1 -- 172.21.15.150:0/2130648763 learned_addr learned my addr 172.21.15.150:0/2130648763 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4eaa00640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4f40cefc0 0x7fd4f41e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3e00640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f41461f0 0x7fd4f415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3400640 1 -- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f41461f0 msgr2=0x7fd4f415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3400640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f41461f0 0x7fd4f415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3400640 1 -- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4f40cefc0 msgr2=0x7fd4f41e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3400640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4f40cefc0 0x7fd4f41e8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3400640 1 -- 172.21.15.150:0/2130648763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f41e7a20 con 0x7fd4f40b3e10 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4f3400640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40b3e10 0x7fd4f41e7420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400cad0 tx=0x7fd4e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4ebe00640 1 -- 172.21.15.150:0/2130648763 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4017070 con 0x7fd4f40b3e10 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4ebe00640 1 -- 172.21.15.150:0/2130648763 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e400ac30 con 0x7fd4f40b3e10 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.928+0000 7fd4ebe00640 1 -- 172.21.15.150:0/2130648763 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4012400 con 0x7fd4f40b3e10 2024-04-15T15:23:55.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.929+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f41e7d00 con 0x7fd4f40b3e10 2024-04-15T15:23:55.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.929+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f41e82c0 con 0x7fd4f40b3e10 2024-04-15T15:23:55.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.930+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4068c80 con 0x7fd4f40b3e10 2024-04-15T15:23:55.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.931+0000 7fd4ebe00640 1 -- 172.21.15.150:0/2130648763 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4e401a070 con 0x7fd4f40b3e10 2024-04-15T15:23:55.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.931+0000 7fd4ebe00640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:55.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.932+0000 7fd4ebe00640 1 -- 172.21.15.150:0/2130648763 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd4e408b910 con 0x7fd4f40b3e10 2024-04-15T15:23:55.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.932+0000 7fd4eaa00640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:55.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.933+0000 7fd4eaa00640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd4d006c8f0 0x7fd4d006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc036000 tx=0x7fd4dc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:55.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:55.936+0000 7fd4ebe00640 1 -- 172.21.15.150:0/2130648763 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4e4059d40 con 0x7fd4f40b3e10 2024-04-15T15:23:56.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.131+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4f415ea60 con 0x7fd4f40b3e10 2024-04-15T15:23:56.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.134+0000 7fd4ebe00640 1 -- 172.21.15.150:0/2130648763 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16666 (secure 0 0 0) 0x7fd4e4059640 con 0x7fd4f40b3e10 2024-04-15T15:23:56.136 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:56.136 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":21,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:54.683131+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:56.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd4d006c8f0 msgr2=0x7fd4d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:56.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd4d006c8f0 0x7fd4d006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc036000 tx=0x7fd4dc008030 comp rx=0 tx=0).stop 2024-04-15T15:23:56.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40b3e10 msgr2=0x7fd4f41e7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40b3e10 0x7fd4f41e7420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400cad0 tx=0x7fd4e400cf90 comp rx=0 tx=0).stop 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 shutdown_connections 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4f41461f0 0x7fd4f415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4f40cefc0 0x7fd4f41e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 --2- 172.21.15.150:0/2130648763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4f40b3e10 0x7fd4f41e7420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.137+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 >> 172.21.15.150:0/2130648763 conn(0x7fd4f40b27b0 msgr2=0x7fd4f415dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.138+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 shutdown_connections 2024-04-15T15:23:56.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.138+0000 7fd4fa000640 1 -- 172.21.15.150:0/2130648763 wait complete. 2024-04-15T15:23:56.156 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-15T15:23:56.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.365+0000 7f7156400640 1 Processor -- start 2024-04-15T15:23:56.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.365+0000 7f7156400640 1 -- start start 2024-04-15T15:23:56.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.367+0000 7f7156400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f715015a440 0x7f715015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:56.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.367+0000 7f7156400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715015d520 con 0x7f7150072150 2024-04-15T15:23:56.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.367+0000 7f7156400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715015d800 con 0x7f715006fc80 2024-04-15T15:23:56.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.367+0000 7f7156400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f715006e280 con 0x7f715015a440 2024-04-15T15:23:56.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.367+0000 7f714fe00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7150072150 0x7f715006e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45904/0 (socket says 172.21.15.150:45904) 2024-04-15T15:23:56.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.367+0000 7f714fe00640 1 -- 172.21.15.150:0/1277165678 learned_addr learned my addr 172.21.15.150:0/1277165678 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:56.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.367+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7597874 0 0) 0x7f715015d520 con 0x7f7150072150 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7134003520 con 0x7f7150072150 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714f400640 1 --2- 172.21.15.150:0/1277165678 >> v2:172.21.15.187:3300/0 conn(0x7f715015a440 0x7f715015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2312415425 0 0) 0x7f7134003520 con 0x7f7150072150 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 >> v2:172.21.15.187:3300/0 conn(0x7f715015a440 msgr2=0x7f715015cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 --2- 172.21.15.150:0/1277165678 >> v2:172.21.15.187:3300/0 conn(0x7f715015a440 0x7f715015cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 >> v1:172.21.15.187:6789/0 conn(0x7f715006fc80 legacy=0x7f71500cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715015dae0 con 0x7f7150072150 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f71480019f0 con 0x7f7150072150 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 >> v1:172.21.15.150:6789/0 conn(0x7f7150072150 legacy=0x7f715006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71340039b0 0x7f7134003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7134004800 0x7f7134004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:56.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7134005210 0x7f713400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f715006e280 con 0x7f7134005210 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f713400c140 con 0x7f7134004800 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f713400c340 con 0x7f71340039b0 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.368+0000 7f714fe00640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7134005210 0x7f713400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714f400640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71340039b0 0x7f7134003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714ea00640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7134004800 0x7f7134004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714fe00640 1 -- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71340039b0 msgr2=0x7f7134003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714fe00640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71340039b0 0x7f7134003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714fe00640 1 -- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7134004800 msgr2=0x7f7134004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714fe00640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7134004800 0x7f7134004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714fe00640 1 -- 172.21.15.150:0/1277165678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7134003520 con 0x7f7134005210 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714ea00640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7134004800 0x7f7134004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:56.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.369+0000 7f714f400640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71340039b0 0x7f7134003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:56.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.370+0000 7f714fe00640 1 -- 172.21.15.150:0/1277165678 reap_dead start 2024-04-15T15:23:56.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.370+0000 7f714fe00640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7134005210 0x7f713400b9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f714800a860 tx=0x7f7134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b483884f7cc8dbd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:56.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.370+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714800a480 con 0x7f7134005210 2024-04-15T15:23:56.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.370+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7148034630 con 0x7f7134005210 2024-04-15T15:23:56.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.370+0000 7f714e000640 1 -- 172.21.15.150:0/1277165678 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148034a20 con 0x7f7134005210 2024-04-15T15:23:56.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.371+0000 7f7156400640 1 -- 172.21.15.150:0/1277165678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7134005210 msgr2=0x7f713400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.371+0000 7f7156400640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7134005210 0x7f713400b9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f714800a860 tx=0x7f7134005830 comp rx=0 tx=0).stop 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.371+0000 7f7156400640 1 -- 172.21.15.150:0/1277165678 shutdown_connections 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.371+0000 7f7156400640 1 --2- 172.21.15.150:0/1277165678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7134005210 0x7f713400b9c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.371+0000 7f7156400640 1 -- 172.21.15.150:0/1277165678 >> 172.21.15.150:0/1277165678 conn(0x7f71500b2460 msgr2=0x7f71500b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.371+0000 7f7156400640 1 -- 172.21.15.150:0/1277165678 shutdown_connections 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.371+0000 7f7156400640 1 -- 172.21.15.150:0/1277165678 wait complete. 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.372+0000 7f7156400640 1 Processor -- start 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.373+0000 7f7156400640 1 -- start start 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.373+0000 7f7156400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7134005210 0x7f715013cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.373+0000 7f7156400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f715012f050 0x7f715012f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.373+0000 7f7156400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71500ae5a0 0x7f71500aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:56.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.373+0000 7f7156400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f715006f510 con 0x7f715012f050 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.373+0000 7f7156400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f715015ddc0 con 0x7f71500ae5a0 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.373+0000 7f7156400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f715015cef0 con 0x7f7134005210 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f715012f050 0x7f715012f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f715012f050 0x7f715012f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59344/0 (socket says 172.21.15.150:59344) 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 -- 172.21.15.150:0/665654475 learned_addr learned my addr 172.21.15.150:0/665654475 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714f400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7134005210 0x7f715013cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714fe00640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71500ae5a0 0x7f71500aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 -- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7134005210 msgr2=0x7f715013cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7134005210 0x7f715013cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 -- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71500ae5a0 msgr2=0x7f71500aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71500ae5a0 0x7f71500aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 -- 172.21.15.150:0/665654475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715013d530 con 0x7f715012f050 2024-04-15T15:23:56.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.374+0000 7f714ea00640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f715012f050 0x7f715012f4d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f713c00d540 tx=0x7f713c00da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:56.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.375+0000 7f7147400640 1 -- 172.21.15.150:0/665654475 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c005480 con 0x7f715012f050 2024-04-15T15:23:56.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.375+0000 7f7147400640 1 -- 172.21.15.150:0/665654475 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713c017070 con 0x7f715012f050 2024-04-15T15:23:56.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.375+0000 7f7147400640 1 -- 172.21.15.150:0/665654475 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c003850 con 0x7f715012f050 2024-04-15T15:23:56.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.375+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715013f680 con 0x7f715012f050 2024-04-15T15:23:56.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.375+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71500adda0 con 0x7f715012f050 2024-04-15T15:23:56.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.376+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7150068c60 con 0x7f715012f050 2024-04-15T15:23:56.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.377+0000 7f7147400640 1 -- 172.21.15.150:0/665654475 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f713c0039f0 con 0x7f715012f050 2024-04-15T15:23:56.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.378+0000 7f7147400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f713006c920 0x7f713006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:56.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.378+0000 7f7147400640 1 -- 172.21.15.150:0/665654475 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f713c08b9b0 con 0x7f715012f050 2024-04-15T15:23:56.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.379+0000 7f714f400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f713006c920 0x7f713006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:56.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.380+0000 7f714f400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f713006c920 0x7f713006ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7140006fd0 tx=0x7f7140008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:56.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.383+0000 7f7147400640 1 -- 172.21.15.150:0/665654475 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f713c059d40 con 0x7f715012f050 2024-04-15T15:23:56.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:56.630+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f715015e0e0 con 0x7f715012f050 2024-04-15T15:23:57.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.287+0000 7f7147400640 1 -- 172.21.15.150:0/665654475 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f713c059640 con 0x7f715012f050 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f713006c920 msgr2=0x7f713006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f713006c920 0x7f713006ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7140006fd0 tx=0x7f7140008040 comp rx=0 tx=0).stop 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f715012f050 msgr2=0x7f715012f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f715012f050 0x7f715012f4d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f713c00d540 tx=0x7f713c00da00 comp rx=0 tx=0).stop 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 shutdown_connections 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71500ae5a0 0x7f71500aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f715012f050 0x7f715012f4d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f713006c920 0x7f713006ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 --2- 172.21.15.150:0/665654475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7134005210 0x7f715013cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 >> 172.21.15.150:0/665654475 conn(0x7f71500b2460 msgr2=0x7f7150072590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:57.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 shutdown_connections 2024-04-15T15:23:57.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.291+0000 7f7156400640 1 -- 172.21.15.150:0/665654475 wait complete. 2024-04-15T15:23:57.313 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-15T15:23:57.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-15T15:23:57.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.528+0000 7fc1cb200640 1 Processor -- start 2024-04-15T15:23:57.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.529+0000 7fc1cb200640 1 -- start start 2024-04-15T15:23:57.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.530+0000 7fc1cb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 0x7fc1cc162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.530+0000 7fc1cb200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc1cc0d2b50 0x7fc1cc0d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.530+0000 7fc1cb200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1cc0b70d0 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.530+0000 7fc1cb200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fc1cc0d34f0 con 0x7fc1cc0d2b50 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.530+0000 7fc1cb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc162dd0 con 0x7fc1cc15c680 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.530+0000 7fc1c9400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc1cc0b6c70 0x7fc1cc15f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33370/0 (socket says 172.21.15.150:33370) 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.530+0000 7fc1c9400640 1 -- 172.21.15.150:0/48356062 learned_addr learned my addr 172.21.15.150:0/48356062 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.531+0000 7fc1ca800640 1 --2- 172.21.15.150:0/48356062 >> v2:172.21.15.150:3300/0 conn(0x7fc1cc0d2b50 0x7fc1cc0d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.531+0000 7fc1c9e00640 1 --2- 172.21.15.150:0/48356062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 0x7fc1cc162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.531+0000 7fc1c8a00640 1 -- 172.21.15.150:0/48356062 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 174465304 0 0) 0x7fc1cc0b70d0 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.531+0000 7fc1c8a00640 1 -- 172.21.15.150:0/48356062 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1b0003520 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.531+0000 7fc1ca800640 1 -- 172.21.15.150:0/48356062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 msgr2=0x7fc1cc162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.531+0000 7fc1ca800640 1 --2- 172.21.15.150:0/48356062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 0x7fc1cc162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.531+0000 7fc1ca800640 1 -- 172.21.15.150:0/48356062 >> v1:172.21.15.150:6789/0 conn(0x7fc1cc0b6c70 legacy=0x7fc1cc15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:57.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.531+0000 7fc1ca800640 1 -- 172.21.15.150:0/48356062 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc1630b0 con 0x7fc1cc0d2b50 2024-04-15T15:23:57.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.532+0000 7fc1ca800640 1 --2- 172.21.15.150:0/48356062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0d2b50 0x7fc1cc0d2fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40097a0 tx=0x7fc1c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7637d48abde852b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:57.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.532+0000 7fc1c8a00640 1 -- 172.21.15.150:0/48356062 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4035020 con 0x7fc1cc0d2b50 2024-04-15T15:23:57.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1c8a00640 1 -- 172.21.15.150:0/48356062 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4031830 con 0x7fc1cc0d2b50 2024-04-15T15:23:57.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1c8a00640 1 -- 172.21.15.150:0/48356062 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4031c00 con 0x7fc1cc0d2b50 2024-04-15T15:23:57.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1cb200640 1 -- 172.21.15.150:0/48356062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0d2b50 msgr2=0x7fc1cc0d2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:57.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1cb200640 1 --2- 172.21.15.150:0/48356062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0d2b50 0x7fc1cc0d2fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40097a0 tx=0x7fc1c402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1cb200640 1 -- 172.21.15.150:0/48356062 shutdown_connections 2024-04-15T15:23:57.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1cb200640 1 --2- 172.21.15.150:0/48356062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0d2b50 0x7fc1cc0d2fb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1cb200640 1 --2- 172.21.15.150:0/48356062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 0x7fc1cc162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1cb200640 1 -- 172.21.15.150:0/48356062 >> 172.21.15.150:0/48356062 conn(0x7fc1cc073480 msgr2=0x7fc1cc073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:57.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.533+0000 7fc1cb200640 1 -- 172.21.15.150:0/48356062 shutdown_connections 2024-04-15T15:23:57.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.534+0000 7fc1cb200640 1 -- 172.21.15.150:0/48356062 wait complete. 2024-04-15T15:23:57.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.535+0000 7fc1cb200640 1 Processor -- start 2024-04-15T15:23:57.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.535+0000 7fc1cb200640 1 -- start start 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1cb200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0b6c70 0x7fc1cc0b9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1cb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1cc0d2b50 0x7fc1cc14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1cb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 0x7fc1cc1f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1cb200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc15ca60 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1cb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc15f330 con 0x7fc1cc0d2b50 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1cb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc15ff60 con 0x7fc1cc15c680 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1c9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0b6c70 0x7fc1cc0b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1c9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0b6c70 0x7fc1cc0b9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44330/0 (socket says 172.21.15.150:44330) 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1c9e00640 1 -- 172.21.15.150:0/636476282 learned_addr learned my addr 172.21.15.150:0/636476282 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1c9e00640 1 -- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 msgr2=0x7fc1cc1f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1c9e00640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 0x7fc1cc1f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1c9e00640 1 -- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1cc0d2b50 msgr2=0x7fc1cc14d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:57.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1c9e00640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1cc0d2b50 0x7fc1cc14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:57.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.537+0000 7fc1c9e00640 1 -- 172.21.15.150:0/636476282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc14e230 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.538+0000 7fc1c9e00640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0b6c70 0x7fc1cc0b9430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc007e40 tx=0x7fc1bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:57.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.538+0000 7fc1c2000640 1 -- 172.21.15.150:0/636476282 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc014620 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.538+0000 7fc1c2000640 1 -- 172.21.15.150:0/636476282 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc019070 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.538+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc1f21e0 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.538+0000 7fc1c2000640 1 -- 172.21.15.150:0/636476282 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc014a00 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.538+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc14dc70 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.539+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1cc0755c0 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.540+0000 7fc1c2000640 1 -- 172.21.15.150:0/636476282 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1bc014ba0 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.541+0000 7fc1c2000640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc1a806c8f0 0x7fc1a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:57.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.541+0000 7fc1c2000640 1 -- 172.21.15.150:0/636476282 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc1bc08c3c0 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.541+0000 7fc1c9400640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc1a806c8f0 0x7fc1a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:57.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.542+0000 7fc1c9400640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc1a806c8f0 0x7fc1a806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000f60 tx=0x7fc1b40078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:57.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.545+0000 7fc1c2000640 1 -- 172.21.15.150:0/636476282 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1bc05a7e0 con 0x7fc1cc0b6c70 2024-04-15T15:23:57.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:57.778+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc1cc162970 con 0x7fc1cc0b6c70 2024-04-15T15:23:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.290+0000 7fc1c2000640 1 -- 172.21.15.150:0/636476282 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fc1bc05a0e0 con 0x7fc1cc0b6c70 2024-04-15T15:23:58.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.293+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc1a806c8f0 msgr2=0x7fc1a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.293+0000 7fc1cb200640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc1a806c8f0 0x7fc1a806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000f60 tx=0x7fc1b40078f0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.293+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0b6c70 msgr2=0x7fc1cc0b9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.293+0000 7fc1cb200640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0b6c70 0x7fc1cc0b9430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc007e40 tx=0x7fc1bc00e4b0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.294+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 shutdown_connections 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.294+0000 7fc1cb200640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc1a806c8f0 0x7fc1a806edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.294+0000 7fc1cb200640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1cc15c680 0x7fc1cc1f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.294+0000 7fc1cb200640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1cc0d2b50 0x7fc1cc14d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.294+0000 7fc1cb200640 1 --2- 172.21.15.150:0/636476282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1cc0b6c70 0x7fc1cc0b9430 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.294+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 >> 172.21.15.150:0/636476282 conn(0x7fc1cc073480 msgr2=0x7fc1cc0d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.294+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 shutdown_connections 2024-04-15T15:23:58.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.294+0000 7fc1cb200640 1 -- 172.21.15.150:0/636476282 wait complete. 2024-04-15T15:23:58.317 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-15T15:23:58.318 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:23:58.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.527+0000 7ff37d800640 1 Processor -- start 2024-04-15T15:23:58.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.527+0000 7ff37d800640 1 -- start start 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.529+0000 7ff37d800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff37815eac0 0x7ff3780d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.529+0000 7ff37d800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff3780d1a10 0x7ff3780b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.529+0000 7ff37d800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3780d2d90 con 0x7ff3780d43c0 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.529+0000 7ff37d800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7ff3780d3070 con 0x7ff37815eac0 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.529+0000 7ff37d800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff3780d3350 con 0x7ff3780d1a10 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.529+0000 7ff376a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff37815eac0 0x7ff3780d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.529+0000 7ff376a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff37815eac0 0x7ff3780d3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44342/0 (socket says 172.21.15.150:44342) 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.530+0000 7ff376a00640 1 -- 172.21.15.150:0/2731601780 learned_addr learned my addr 172.21.15.150:0/2731601780 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:58.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.530+0000 7ff377400640 1 --2- 172.21.15.150:0/2731601780 >> v2:172.21.15.187:3300/0 conn(0x7ff3780d1a10 0x7ff3780b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:58.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.530+0000 7ff376a00640 1 -- 172.21.15.150:0/2731601780 >> v2:172.21.15.187:3300/0 conn(0x7ff3780d1a10 msgr2=0x7ff3780b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:58.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.530+0000 7ff376a00640 1 --2- 172.21.15.150:0/2731601780 >> v2:172.21.15.187:3300/0 conn(0x7ff3780d1a10 0x7ff3780b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.530+0000 7ff376a00640 1 -- 172.21.15.150:0/2731601780 >> v1:172.21.15.187:6789/0 conn(0x7ff3780d43c0 legacy=0x7ff3780d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:58.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.530+0000 7ff376a00640 1 -- 172.21.15.150:0/2731601780 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3780d3630 con 0x7ff37815eac0 2024-04-15T15:23:58.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.531+0000 7ff376a00640 1 --2- 172.21.15.150:0/2731601780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff37815eac0 0x7ff3780d3e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff37806e000 tx=0x7ff36802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ea1ef2a5855b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:58.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff376000640 1 -- 172.21.15.150:0/2731601780 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368034070 con 0x7ff37815eac0 2024-04-15T15:23:58.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff376000640 1 -- 172.21.15.150:0/2731601780 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff368030860 con 0x7ff37815eac0 2024-04-15T15:23:58.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff376000640 1 -- 172.21.15.150:0/2731601780 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368030c30 con 0x7ff37815eac0 2024-04-15T15:23:58.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff37d800640 1 -- 172.21.15.150:0/2731601780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff37815eac0 msgr2=0x7ff3780d3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:58.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff37d800640 1 --2- 172.21.15.150:0/2731601780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff37815eac0 0x7ff3780d3e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff37806e000 tx=0x7ff36802b9a0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff37d800640 1 -- 172.21.15.150:0/2731601780 shutdown_connections 2024-04-15T15:23:58.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff37d800640 1 --2- 172.21.15.150:0/2731601780 >> v2:172.21.15.187:3300/0 conn(0x7ff3780d1a10 0x7ff3780b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff37d800640 1 --2- 172.21.15.150:0/2731601780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff37815eac0 0x7ff3780d3e80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.532+0000 7ff37d800640 1 -- 172.21.15.150:0/2731601780 >> 172.21.15.150:0/2731601780 conn(0x7ff378073690 msgr2=0x7ff37806cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:58.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.533+0000 7ff37d800640 1 -- 172.21.15.150:0/2731601780 shutdown_connections 2024-04-15T15:23:58.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.533+0000 7ff37d800640 1 -- 172.21.15.150:0/2731601780 wait complete. 2024-04-15T15:23:58.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.534+0000 7ff37d800640 1 Processor -- start 2024-04-15T15:23:58.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.534+0000 7ff37d800640 1 -- start start 2024-04-15T15:23:58.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.535+0000 7ff37d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780d1a10 0x7ff378155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:58.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.535+0000 7ff37d800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3780d43c0 0x7ff3781614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:58.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.535+0000 7ff37d800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815eac0 0x7ff378161a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:58.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.535+0000 7ff37d800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff3780d29f0 con 0x7ff3780d43c0 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.535+0000 7ff37d800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff37815eea0 con 0x7ff37815eac0 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.535+0000 7ff37d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff3780d2b70 con 0x7ff3780d1a10 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3780d43c0 0x7ff3781614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3780d43c0 0x7ff3781614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44348/0 (socket says 172.21.15.150:44348) 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 -- 172.21.15.150:0/1661653542 learned_addr learned my addr 172.21.15.150:0/1661653542 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff376a00640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780d1a10 0x7ff378155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff377400640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815eac0 0x7ff378161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 -- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780d1a10 msgr2=0x7ff378155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780d1a10 0x7ff378155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 -- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815eac0 msgr2=0x7ff378161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815eac0 0x7ff378161a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 -- 172.21.15.150:0/1661653542 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3781f55f0 con 0x7ff3780d43c0 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff376a00640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780d1a10 0x7ff378155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:23:58.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36fe00640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3780d43c0 0x7ff3781614e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c400 tx=0x7ff36000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:58.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.536+0000 7ff36ea00640 1 -- 172.21.15.150:0/1661653542 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600052d0 con 0x7ff3780d43c0 2024-04-15T15:23:58.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.537+0000 7ff36ea00640 1 -- 172.21.15.150:0/1661653542 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff360005490 con 0x7ff3780d43c0 2024-04-15T15:23:58.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.537+0000 7ff36ea00640 1 -- 172.21.15.150:0/1661653542 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360007260 con 0x7ff3780d43c0 2024-04-15T15:23:58.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.537+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3781f5bb0 con 0x7ff3780d43c0 2024-04-15T15:23:58.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.537+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3781f6170 con 0x7ff3780d43c0 2024-04-15T15:23:58.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.538+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3781f59c0 con 0x7ff3780d43c0 2024-04-15T15:23:58.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.539+0000 7ff36ea00640 1 -- 172.21.15.150:0/1661653542 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff360007400 con 0x7ff3780d43c0 2024-04-15T15:23:58.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.539+0000 7ff36ea00640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff35406c8f0 0x7ff35406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:58.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.540+0000 7ff36ea00640 1 -- 172.21.15.150:0/1661653542 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff36008b050 con 0x7ff3780d43c0 2024-04-15T15:23:58.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.540+0000 7ff376a00640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff35406c8f0 0x7ff35406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:58.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.541+0000 7ff376a00640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff35406c8f0 0x7ff35406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff368036000 tx=0x7ff36803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:58.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.544+0000 7ff36ea00640 1 -- 172.21.15.150:0/1661653542 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff360059510 con 0x7ff3780d43c0 2024-04-15T15:23:58.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.764+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff378068c80 con 0x7ff3780d43c0 2024-04-15T15:23:58.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.765+0000 7ff36ea00640 1 -- 172.21.15.150:0/1661653542 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7ff360058e10 con 0x7ff3780d43c0 2024-04-15T15:23:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:58.767 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:23:54.683053+0000","modified":"2024-04-15T15:23:57.779875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4250,"qdb_cluster":[4250]},"id":1}]} 2024-04-15T15:23:58.767 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-04-15T15:23:58.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.767+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff35406c8f0 msgr2=0x7ff35406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:58.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.767+0000 7ff37d800640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff35406c8f0 0x7ff35406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff368036000 tx=0x7ff36803b040 comp rx=0 tx=0).stop 2024-04-15T15:23:58.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.767+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3780d43c0 msgr2=0x7ff3781614e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:58.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.767+0000 7ff37d800640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3780d43c0 0x7ff3781614e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c400 tx=0x7ff36000c8c0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.768+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 shutdown_connections 2024-04-15T15:23:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.768+0000 7ff37d800640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815eac0 0x7ff378161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.768+0000 7ff37d800640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3780d43c0 0x7ff3781614e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.768+0000 7ff37d800640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff35406c8f0 0x7ff35406edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.768+0000 7ff37d800640 1 --2- 172.21.15.150:0/1661653542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780d1a10 0x7ff378155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.768+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 >> 172.21.15.150:0/1661653542 conn(0x7ff378073690 msgr2=0x7ff3780ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.768+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 shutdown_connections 2024-04-15T15:23:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.768+0000 7ff37d800640 1 -- 172.21.15.150:0/1661653542 wait complete. 2024-04-15T15:23:58.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:23:58.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.996+0000 7effed800640 1 Processor -- start 2024-04-15T15:23:58.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.996+0000 7effed800640 1 -- start start 2024-04-15T15:23:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.997+0000 7effed800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7effe815d2f0 0x7effe806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.997+0000 7effed800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe815dbb0 con 0x7effe8072660 2024-04-15T15:23:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.997+0000 7effed800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe815d8d0 con 0x7effe80cee00 2024-04-15T15:23:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.997+0000 7effed800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7effe806f4f0 con 0x7effe815d2f0 2024-04-15T15:23:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.997+0000 7effdf400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7effe8072660 0x7effe815cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33376/0 (socket says 172.21.15.150:33376) 2024-04-15T15:23:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.997+0000 7effdf400640 1 -- 172.21.15.150:0/2178767163 learned_addr learned my addr 172.21.15.150:0/2178767163 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:58.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.997+0000 7effece00640 1 --2- 172.21.15.150:0/2178767163 >> v2:172.21.15.150:3300/0 conn(0x7effe815d2f0 0x7effe806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.998+0000 7effe7400640 1 -- 172.21.15.150:0/2178767163 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125841795 0 0) 0x7effe815dbb0 con 0x7effe8072660 2024-04-15T15:23:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.998+0000 7effe7400640 1 -- 172.21.15.150:0/2178767163 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effcc003520 con 0x7effe8072660 2024-04-15T15:23:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.998+0000 7effe7400640 1 -- 172.21.15.150:0/2178767163 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3145686236 0 0) 0x7effcc003520 con 0x7effe8072660 2024-04-15T15:23:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.998+0000 7effece00640 1 -- 172.21.15.150:0/2178767163 >> v1:172.21.15.187:6789/0 conn(0x7effe80cee00 legacy=0x7effe8071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.998+0000 7effece00640 1 -- 172.21.15.150:0/2178767163 >> v1:172.21.15.150:6789/0 conn(0x7effe8072660 legacy=0x7effe815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:58.998+0000 7effece00640 1 -- 172.21.15.150:0/2178767163 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe806dde0 con 0x7effe815d2f0 2024-04-15T15:23:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.000+0000 7effece00640 1 --2- 172.21.15.150:0/2178767163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe806e8c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7effe0009d90 tx=0x7effe002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4167fe469f2edd91 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.000+0000 7effe7400640 1 -- 172.21.15.150:0/2178767163 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0039070 con 0x7effe815d2f0 2024-04-15T15:23:59.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.000+0000 7effe7400640 1 -- 172.21.15.150:0/2178767163 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe002c940 con 0x7effe815d2f0 2024-04-15T15:23:59.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.000+0000 7effe7400640 1 -- 172.21.15.150:0/2178767163 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0001670 con 0x7effe815d2f0 2024-04-15T15:23:59.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.001+0000 7effed800640 1 -- 172.21.15.150:0/2178767163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 msgr2=0x7effe806e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.001+0000 7effed800640 1 --2- 172.21.15.150:0/2178767163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe806e8c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7effe0009d90 tx=0x7effe002be10 comp rx=0 tx=0).stop 2024-04-15T15:23:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.001+0000 7effed800640 1 -- 172.21.15.150:0/2178767163 shutdown_connections 2024-04-15T15:23:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.001+0000 7effed800640 1 --2- 172.21.15.150:0/2178767163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe806e8c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.001+0000 7effed800640 1 -- 172.21.15.150:0/2178767163 >> 172.21.15.150:0/2178767163 conn(0x7effe806b930 msgr2=0x7effe806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.001+0000 7effed800640 1 -- 172.21.15.150:0/2178767163 shutdown_connections 2024-04-15T15:23:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.001+0000 7effed800640 1 -- 172.21.15.150:0/2178767163 wait complete. 2024-04-15T15:23:59.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.002+0000 7effed800640 1 Processor -- start 2024-04-15T15:23:59.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.003+0000 7effed800640 1 -- start start 2024-04-15T15:23:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effed800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe8072660 0x7effe8147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effed800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe80cee00 0x7effe8160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effed800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effed800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7effe815dda0 con 0x7effe815d2f0 2024-04-15T15:23:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effed800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7effe806e3a0 con 0x7effe8072660 2024-04-15T15:23:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effed800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7effe815df20 con 0x7effe80cee00 2024-04-15T15:23:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effece00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effece00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe815a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44372/0 (socket says 172.21.15.150:44372) 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effece00640 1 -- 172.21.15.150:0/1830716163 learned_addr learned my addr 172.21.15.150:0/1830716163 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effe7e00640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe8072660 0x7effe8147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effdf400640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe80cee00 0x7effe8160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effece00640 1 -- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe80cee00 msgr2=0x7effe8160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effece00640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe80cee00 0x7effe8160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effece00640 1 -- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe8072660 msgr2=0x7effe8147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effece00640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe8072660 0x7effe8147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.004+0000 7effece00640 1 -- 172.21.15.150:0/1830716163 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe8152880 con 0x7effe815d2f0 2024-04-15T15:23:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.005+0000 7effece00640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe815a7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7effe0009eb0 tx=0x7effe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.005+0000 7effe4c00640 1 -- 172.21.15.150:0/1830716163 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0044020 con 0x7effe815d2f0 2024-04-15T15:23:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.005+0000 7effe4c00640 1 -- 172.21.15.150:0/1830716163 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe0004240 con 0x7effe815d2f0 2024-04-15T15:23:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.005+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe8151d00 con 0x7effe815d2f0 2024-04-15T15:23:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.005+0000 7effe4c00640 1 -- 172.21.15.150:0/1830716163 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0039040 con 0x7effe815d2f0 2024-04-15T15:23:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.005+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe81522c0 con 0x7effe815d2f0 2024-04-15T15:23:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.006+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe80755c0 con 0x7effe815d2f0 2024-04-15T15:23:59.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.007+0000 7effe4c00640 1 -- 172.21.15.150:0/1830716163 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effe0047020 con 0x7effe815d2f0 2024-04-15T15:23:59.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.008+0000 7effe4c00640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7effc406c8f0 0x7effc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.008+0000 7effe4c00640 1 -- 172.21.15.150:0/1830716163 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7effe00adfb0 con 0x7effe815d2f0 2024-04-15T15:23:59.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.009+0000 7effe7e00640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7effc406c8f0 0x7effc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.010+0000 7effe7e00640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7effc406c8f0 0x7effc406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7effd8000f60 tx=0x7effd8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.013+0000 7effe4c00640 1 -- 172.21.15.150:0/1830716163 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effe004e070 con 0x7effe815d2f0 2024-04-15T15:23:59.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.216+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effe8072230 con 0x7effe815d2f0 2024-04-15T15:23:59.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.218+0000 7effe4c00640 1 -- 172.21.15.150:0/1830716163 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7effe0032340 con 0x7effe815d2f0 2024-04-15T15:23:59.220 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:59.220 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7effc406c8f0 msgr2=0x7effc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7effc406c8f0 0x7effc406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7effd8000f60 tx=0x7effd8009040 comp rx=0 tx=0).stop 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 msgr2=0x7effe815a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe815a7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7effe0009eb0 tx=0x7effe0004a40 comp rx=0 tx=0).stop 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 shutdown_connections 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effe815d2f0 0x7effe815a7e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7effc406c8f0 0x7effc406edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe80cee00 0x7effe8160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 --2- 172.21.15.150:0/1830716163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe8072660 0x7effe8147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 >> 172.21.15.150:0/1830716163 conn(0x7effe806b930 msgr2=0x7effe815ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 shutdown_connections 2024-04-15T15:23:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.221+0000 7effed800640 1 -- 172.21.15.150:0/1830716163 wait complete. 2024-04-15T15:23:59.243 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-15T15:23:59.243 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:23:59.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.443+0000 7f6eeac00640 1 Processor -- start 2024-04-15T15:23:59.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.443+0000 7f6eeac00640 1 -- start start 2024-04-15T15:23:59.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6eeac00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ee40cee00 0x7f6ee406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6eeac00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ee4071f50 0x7f6ee40723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6eeac00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee415dbb0 con 0x7f6ee415cbe0 2024-04-15T15:23:59.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6eeac00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee406f4f0 con 0x7f6ee40cee00 2024-04-15T15:23:59.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6eeac00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee40728f0 con 0x7f6ee4071f50 2024-04-15T15:23:59.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6ee9800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ee40cee00 0x7f6ee406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6eea200640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6ee415cbe0 0x7f6ee415d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33388/0 (socket says 172.21.15.150:33388) 2024-04-15T15:23:59.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6eea200640 1 -- 172.21.15.150:0/1420249413 learned_addr learned my addr 172.21.15.150:0/1420249413 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:59.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.445+0000 7f6ee8e00640 1 --2- 172.21.15.150:0/1420249413 >> v2:172.21.15.187:3300/0 conn(0x7f6ee4071f50 0x7f6ee40723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.446+0000 7f6ee9800640 1 -- 172.21.15.150:0/1420249413 >> v2:172.21.15.187:3300/0 conn(0x7f6ee4071f50 msgr2=0x7f6ee40723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:59.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.446+0000 7f6ee9800640 1 --2- 172.21.15.150:0/1420249413 >> v2:172.21.15.187:3300/0 conn(0x7f6ee4071f50 0x7f6ee40723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.446+0000 7f6ee9800640 1 -- 172.21.15.150:0/1420249413 >> v1:172.21.15.150:6789/0 conn(0x7f6ee415cbe0 legacy=0x7f6ee415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.446+0000 7f6ee9800640 1 -- 172.21.15.150:0/1420249413 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee415d8d0 con 0x7f6ee40cee00 2024-04-15T15:23:59.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.447+0000 7f6ee9800640 1 --2- 172.21.15.150:0/1420249413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee406e8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40b5aa0 tx=0x7f6ed802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17a432cbdb9e330 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.447+0000 7f6ee3e00640 1 -- 172.21.15.150:0/1420249413 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8035020 con 0x7f6ee40cee00 2024-04-15T15:23:59.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.447+0000 7f6ee3e00640 1 -- 172.21.15.150:0/1420249413 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8031830 con 0x7f6ee40cee00 2024-04-15T15:23:59.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.447+0000 7f6ee3e00640 1 -- 172.21.15.150:0/1420249413 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8031c00 con 0x7f6ee40cee00 2024-04-15T15:23:59.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.448+0000 7f6eeac00640 1 -- 172.21.15.150:0/1420249413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 msgr2=0x7f6ee406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.448+0000 7f6eeac00640 1 --2- 172.21.15.150:0/1420249413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee406e8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40b5aa0 tx=0x7f6ed802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.448+0000 7f6eeac00640 1 -- 172.21.15.150:0/1420249413 shutdown_connections 2024-04-15T15:23:59.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.448+0000 7f6eeac00640 1 --2- 172.21.15.150:0/1420249413 >> v2:172.21.15.187:3300/0 conn(0x7f6ee4071f50 0x7f6ee40723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.448+0000 7f6eeac00640 1 --2- 172.21.15.150:0/1420249413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee406e8c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.448+0000 7f6eeac00640 1 -- 172.21.15.150:0/1420249413 >> 172.21.15.150:0/1420249413 conn(0x7f6ee406b930 msgr2=0x7f6ee406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:59.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.448+0000 7f6eeac00640 1 -- 172.21.15.150:0/1420249413 shutdown_connections 2024-04-15T15:23:59.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.448+0000 7f6eeac00640 1 -- 172.21.15.150:0/1420249413 wait complete. 2024-04-15T15:23:59.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.450+0000 7f6eeac00640 1 Processor -- start 2024-04-15T15:23:59.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.450+0000 7f6eeac00640 1 -- start start 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6eeac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ee4071f50 0x7f6ee4149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6eeac00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee4158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6eeac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ee415cbe0 0x7f6ee414a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6eeac00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee406e3a0 con 0x7f6ee40cee00 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6eeac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee406e0c0 con 0x7f6ee4071f50 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6eeac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee415dda0 con 0x7f6ee415cbe0 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee4158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee4158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44392/0 (socket says 172.21.15.150:44392) 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 -- 172.21.15.150:0/3808647895 learned_addr learned my addr 172.21.15.150:0/3808647895 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 -- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ee415cbe0 msgr2=0x7f6ee414a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ee415cbe0 0x7f6ee414a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 -- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ee4071f50 msgr2=0x7f6ee4149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ee4071f50 0x7f6ee4149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 -- 172.21.15.150:0/3808647895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee414a490 con 0x7f6ee40cee00 2024-04-15T15:23:59.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.451+0000 7f6ee8e00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee4158820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400d800 tx=0x7f6ed400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.452+0000 7f6ee1600640 1 -- 172.21.15.150:0/3808647895 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400b6c0 con 0x7f6ee40cee00 2024-04-15T15:23:59.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.452+0000 7f6ee1600640 1 -- 172.21.15.150:0/3808647895 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed4019070 con 0x7f6ee40cee00 2024-04-15T15:23:59.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.452+0000 7f6ee1600640 1 -- 172.21.15.150:0/3808647895 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400ba80 con 0x7f6ee40cee00 2024-04-15T15:23:59.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.452+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee4156f80 con 0x7f6ee40cee00 2024-04-15T15:23:59.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.452+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee4157540 con 0x7f6ee40cee00 2024-04-15T15:23:59.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.453+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee40730d0 con 0x7f6ee40cee00 2024-04-15T15:23:59.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.454+0000 7f6ee1600640 1 -- 172.21.15.150:0/3808647895 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ed400bc20 con 0x7f6ee40cee00 2024-04-15T15:23:59.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.455+0000 7f6ee1600640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6ec806c8f0 0x7f6ec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.455+0000 7f6ee1600640 1 -- 172.21.15.150:0/3808647895 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6ed408c0e0 con 0x7f6ee40cee00 2024-04-15T15:23:59.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.456+0000 7f6ee9800640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6ec806c8f0 0x7f6ec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.457+0000 7f6ee9800640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6ec806c8f0 0x7f6ec806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8036000 tx=0x7f6ed8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.460+0000 7f6ee1600640 1 -- 172.21.15.150:0/3808647895 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ee40730d0 con 0x7f6ee40cee00 2024-04-15T15:23:59.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.659+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ee40cdee0 con 0x7f6ec806c8f0 2024-04-15T15:23:59.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.681+0000 7f6ee1600640 1 -- 172.21.15.150:0/3808647895 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f6ee40cdee0 con 0x7f6ec806c8f0 2024-04-15T15:23:59.687 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:23:59.688 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.687+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6ec806c8f0 msgr2=0x7f6ec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.687+0000 7f6eeac00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6ec806c8f0 0x7f6ec806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8036000 tx=0x7f6ed8030000 comp rx=0 tx=0).stop 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.687+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 msgr2=0x7f6ee4158820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.687+0000 7f6eeac00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee4158820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400d800 tx=0x7f6ed400dcc0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.688+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 shutdown_connections 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.688+0000 7f6eeac00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6ec806c8f0 0x7f6ec806edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.688+0000 7f6eeac00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ee415cbe0 0x7f6ee414a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.688+0000 7f6eeac00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee40cee00 0x7f6ee4158820 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.688+0000 7f6eeac00640 1 --2- 172.21.15.150:0/3808647895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ee4071f50 0x7f6ee4149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.688+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 >> 172.21.15.150:0/3808647895 conn(0x7f6ee406b930 msgr2=0x7f6ee40b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.688+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 shutdown_connections 2024-04-15T15:23:59.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.688+0000 7f6eeac00640 1 -- 172.21.15.150:0/3808647895 wait complete. 2024-04-15T15:23:59.710 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-15T15:23:59.217450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219148,"kb_used_data":2940,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755572,"statfs":{"total":773094113280,"available":772869705728,"internally_reserved":0,"allocated":3010560,"data_stored":1789528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002052"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418140+0000","last_change":"2024-04-15T15:23:55.263891+0000","last_active":"2024-04-15T15:23:56.418140+0000","last_peered":"2024-04-15T15:23:56.418140+0000","last_clean":"2024-04-15T15:23:56.418140+0000","last_became_active":"2024-04-15T15:23:55.263577+0000","last_became_peered":"2024-04-15T15:23:55.263577+0000","last_unstale":"2024-04-15T15:23:56.418140+0000","last_undegraded":"2024-04-15T15:23:56.418140+0000","last_fullsized":"2024-04-15T15:23:56.418140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:58.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255982+0000","last_change":"2024-04-15T15:23:55.259842+0000","last_active":"2024-04-15T15:23:56.255982+0000","last_peered":"2024-04-15T15:23:56.255982+0000","last_clean":"2024-04-15T15:23:56.255982+0000","last_became_active":"2024-04-15T15:23:55.259375+0000","last_became_peered":"2024-04-15T15:23:55.259375+0000","last_unstale":"2024-04-15T15:23:56.255982+0000","last_undegraded":"2024-04-15T15:23:56.255982+0000","last_fullsized":"2024-04-15T15:23:56.255982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418953+0000","last_change":"2024-04-15T15:23:55.249464+0000","last_active":"2024-04-15T15:23:56.418953+0000","last_peered":"2024-04-15T15:23:56.418953+0000","last_clean":"2024-04-15T15:23:56.418953+0000","last_became_active":"2024-04-15T15:23:55.248697+0000","last_became_peered":"2024-04-15T15:23:55.248697+0000","last_unstale":"2024-04-15T15:23:56.418953+0000","last_undegraded":"2024-04-15T15:23:56.418953+0000","last_fullsized":"2024-04-15T15:23:56.418953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262156+0000","last_change":"2024-04-15T15:23:55.267985+0000","last_active":"2024-04-15T15:23:56.262156+0000","last_peered":"2024-04-15T15:23:56.262156+0000","last_clean":"2024-04-15T15:23:56.262156+0000","last_became_active":"2024-04-15T15:23:55.267575+0000","last_became_peered":"2024-04-15T15:23:55.267575+0000","last_unstale":"2024-04-15T15:23:56.262156+0000","last_undegraded":"2024-04-15T15:23:56.262156+0000","last_fullsized":"2024-04-15T15:23:56.262156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:11.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419665+0000","last_change":"2024-04-15T15:23:55.260235+0000","last_active":"2024-04-15T15:23:56.419665+0000","last_peered":"2024-04-15T15:23:56.419665+0000","last_clean":"2024-04-15T15:23:56.419665+0000","last_became_active":"2024-04-15T15:23:55.259947+0000","last_became_peered":"2024-04-15T15:23:55.259947+0000","last_unstale":"2024-04-15T15:23:56.419665+0000","last_undegraded":"2024-04-15T15:23:56.419665+0000","last_fullsized":"2024-04-15T15:23:56.419665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:18:41.223608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261963+0000","last_change":"2024-04-15T15:23:55.257936+0000","last_active":"2024-04-15T15:23:56.261963+0000","last_peered":"2024-04-15T15:23:56.261963+0000","last_clean":"2024-04-15T15:23:56.261963+0000","last_became_active":"2024-04-15T15:23:55.257397+0000","last_became_peered":"2024-04-15T15:23:55.257397+0000","last_unstale":"2024-04-15T15:23:56.261963+0000","last_undegraded":"2024-04-15T15:23:56.261963+0000","last_fullsized":"2024-04-15T15:23:56.261963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417763+0000","last_change":"2024-04-15T15:23:55.265207+0000","last_active":"2024-04-15T15:23:56.417763+0000","last_peered":"2024-04-15T15:23:56.417763+0000","last_clean":"2024-04-15T15:23:56.417763+0000","last_became_active":"2024-04-15T15:23:55.264857+0000","last_became_peered":"2024-04-15T15:23:55.264857+0000","last_unstale":"2024-04-15T15:23:56.417763+0000","last_undegraded":"2024-04-15T15:23:56.417763+0000","last_fullsized":"2024-04-15T15:23:56.417763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:35.342060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416683+0000","last_change":"2024-04-15T15:23:55.270590+0000","last_active":"2024-04-15T15:23:56.416683+0000","last_peered":"2024-04-15T15:23:56.416683+0000","last_clean":"2024-04-15T15:23:56.416683+0000","last_became_active":"2024-04-15T15:23:55.267976+0000","last_became_peered":"2024-04-15T15:23:55.267976+0000","last_unstale":"2024-04-15T15:23:56.416683+0000","last_undegraded":"2024-04-15T15:23:56.416683+0000","last_fullsized":"2024-04-15T15:23:56.416683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:11.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418517+0000","last_change":"2024-04-15T15:23:55.274056+0000","last_active":"2024-04-15T15:23:56.418517+0000","last_peered":"2024-04-15T15:23:56.418517+0000","last_clean":"2024-04-15T15:23:56.418517+0000","last_became_active":"2024-04-15T15:23:55.273746+0000","last_became_peered":"2024-04-15T15:23:55.273746+0000","last_unstale":"2024-04-15T15:23:56.418517+0000","last_undegraded":"2024-04-15T15:23:56.418517+0000","last_fullsized":"2024-04-15T15:23:56.418517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:31:11.922705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262425+0000","last_change":"2024-04-15T15:23:55.272281+0000","last_active":"2024-04-15T15:23:56.262425+0000","last_peered":"2024-04-15T15:23:56.262425+0000","last_clean":"2024-04-15T15:23:56.262425+0000","last_became_active":"2024-04-15T15:23:55.269795+0000","last_became_peered":"2024-04-15T15:23:55.269795+0000","last_unstale":"2024-04-15T15:23:56.262425+0000","last_undegraded":"2024-04-15T15:23:56.262425+0000","last_fullsized":"2024-04-15T15:23:56.262425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263663+0000","last_change":"2024-04-15T15:23:55.269199+0000","last_active":"2024-04-15T15:23:56.263663+0000","last_peered":"2024-04-15T15:23:56.263663+0000","last_clean":"2024-04-15T15:23:56.263663+0000","last_became_active":"2024-04-15T15:23:55.268870+0000","last_became_peered":"2024-04-15T15:23:55.268870+0000","last_unstale":"2024-04-15T15:23:56.263663+0000","last_undegraded":"2024-04-15T15:23:56.263663+0000","last_fullsized":"2024-04-15T15:23:56.263663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418307+0000","last_change":"2024-04-15T15:23:55.263524+0000","last_active":"2024-04-15T15:23:56.418307+0000","last_peered":"2024-04-15T15:23:56.418307+0000","last_clean":"2024-04-15T15:23:56.418307+0000","last_became_active":"2024-04-15T15:23:55.263210+0000","last_became_peered":"2024-04-15T15:23:55.263210+0000","last_unstale":"2024-04-15T15:23:56.418307+0000","last_undegraded":"2024-04-15T15:23:56.418307+0000","last_fullsized":"2024-04-15T15:23:56.418307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:32.154491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415704+0000","last_change":"2024-04-15T15:23:55.272996+0000","last_active":"2024-04-15T15:23:56.415704+0000","last_peered":"2024-04-15T15:23:56.415704+0000","last_clean":"2024-04-15T15:23:56.415704+0000","last_became_active":"2024-04-15T15:23:55.272701+0000","last_became_peered":"2024-04-15T15:23:55.272701+0000","last_unstale":"2024-04-15T15:23:56.415704+0000","last_undegraded":"2024-04-15T15:23:56.415704+0000","last_fullsized":"2024-04-15T15:23:56.415704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417462+0000","last_change":"2024-04-15T15:23:55.260534+0000","last_active":"2024-04-15T15:23:56.417462+0000","last_peered":"2024-04-15T15:23:56.417462+0000","last_clean":"2024-04-15T15:23:56.417462+0000","last_became_active":"2024-04-15T15:23:55.258506+0000","last_became_peered":"2024-04-15T15:23:55.258506+0000","last_unstale":"2024-04-15T15:23:56.417462+0000","last_undegraded":"2024-04-15T15:23:56.417462+0000","last_fullsized":"2024-04-15T15:23:56.417462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255395+0000","last_change":"2024-04-15T15:23:55.265936+0000","last_active":"2024-04-15T15:23:56.255395+0000","last_peered":"2024-04-15T15:23:56.255395+0000","last_clean":"2024-04-15T15:23:56.255395+0000","last_became_active":"2024-04-15T15:23:55.265118+0000","last_became_peered":"2024-04-15T15:23:55.265118+0000","last_unstale":"2024-04-15T15:23:56.255395+0000","last_undegraded":"2024-04-15T15:23:56.255395+0000","last_fullsized":"2024-04-15T15:23:56.255395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.260985+0000","last_change":"2024-04-15T15:23:55.252289+0000","last_active":"2024-04-15T15:23:56.260985+0000","last_peered":"2024-04-15T15:23:56.260985+0000","last_clean":"2024-04-15T15:23:56.260985+0000","last_became_active":"2024-04-15T15:23:55.252007+0000","last_became_peered":"2024-04-15T15:23:55.252007+0000","last_unstale":"2024-04-15T15:23:56.260985+0000","last_undegraded":"2024-04-15T15:23:56.260985+0000","last_fullsized":"2024-04-15T15:23:56.260985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262956+0000","last_change":"2024-04-15T15:23:55.256170+0000","last_active":"2024-04-15T15:23:56.262956+0000","last_peered":"2024-04-15T15:23:56.262956+0000","last_clean":"2024-04-15T15:23:56.262956+0000","last_became_active":"2024-04-15T15:23:55.255896+0000","last_became_peered":"2024-04-15T15:23:55.255896+0000","last_unstale":"2024-04-15T15:23:56.262956+0000","last_undegraded":"2024-04-15T15:23:56.262956+0000","last_fullsized":"2024-04-15T15:23:56.262956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:16:50.418929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417610+0000","last_change":"2024-04-15T15:23:55.269766+0000","last_active":"2024-04-15T15:23:56.417610+0000","last_peered":"2024-04-15T15:23:56.417610+0000","last_clean":"2024-04-15T15:23:56.417610+0000","last_became_active":"2024-04-15T15:23:55.269443+0000","last_became_peered":"2024-04-15T15:23:55.269443+0000","last_unstale":"2024-04-15T15:23:56.417610+0000","last_undegraded":"2024-04-15T15:23:56.417610+0000","last_fullsized":"2024-04-15T15:23:56.417610+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:43:55.932310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419019+0000","last_change":"2024-04-15T15:23:55.265016+0000","last_active":"2024-04-15T15:23:56.419019+0000","last_peered":"2024-04-15T15:23:56.419019+0000","last_clean":"2024-04-15T15:23:56.419019+0000","last_became_active":"2024-04-15T15:23:55.258660+0000","last_became_peered":"2024-04-15T15:23:55.258660+0000","last_unstale":"2024-04-15T15:23:56.419019+0000","last_undegraded":"2024-04-15T15:23:56.419019+0000","last_fullsized":"2024-04-15T15:23:56.419019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:08:08.360698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262025+0000","last_change":"2024-04-15T15:23:55.275816+0000","last_active":"2024-04-15T15:23:56.262025+0000","last_peered":"2024-04-15T15:23:56.262025+0000","last_clean":"2024-04-15T15:23:56.262025+0000","last_became_active":"2024-04-15T15:23:55.274960+0000","last_became_peered":"2024-04-15T15:23:55.274960+0000","last_unstale":"2024-04-15T15:23:56.262025+0000","last_undegraded":"2024-04-15T15:23:56.262025+0000","last_fullsized":"2024-04-15T15:23:56.262025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417084+0000","last_change":"2024-04-15T15:23:55.267469+0000","last_active":"2024-04-15T15:23:56.417084+0000","last_peered":"2024-04-15T15:23:56.417084+0000","last_clean":"2024-04-15T15:23:56.417084+0000","last_became_active":"2024-04-15T15:23:55.266878+0000","last_became_peered":"2024-04-15T15:23:55.266878+0000","last_unstale":"2024-04-15T15:23:56.417084+0000","last_undegraded":"2024-04-15T15:23:56.417084+0000","last_fullsized":"2024-04-15T15:23:56.417084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262363+0000","last_change":"2024-04-15T15:23:55.262231+0000","last_active":"2024-04-15T15:23:56.262363+0000","last_peered":"2024-04-15T15:23:56.262363+0000","last_clean":"2024-04-15T15:23:56.262363+0000","last_became_active":"2024-04-15T15:23:55.261861+0000","last_became_peered":"2024-04-15T15:23:55.261861+0000","last_unstale":"2024-04-15T15:23:56.262363+0000","last_undegraded":"2024-04-15T15:23:56.262363+0000","last_fullsized":"2024-04-15T15:23:56.262363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:38.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417901+0000","last_change":"2024-04-15T15:23:55.275456+0000","last_active":"2024-04-15T15:23:56.417901+0000","last_peered":"2024-04-15T15:23:56.417901+0000","last_clean":"2024-04-15T15:23:56.417901+0000","last_became_active":"2024-04-15T15:23:55.275108+0000","last_became_peered":"2024-04-15T15:23:55.275108+0000","last_unstale":"2024-04-15T15:23:56.417901+0000","last_undegraded":"2024-04-15T15:23:56.417901+0000","last_fullsized":"2024-04-15T15:23:56.417901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:38.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261274+0000","last_change":"2024-04-15T15:23:55.270445+0000","last_active":"2024-04-15T15:23:56.261274+0000","last_peered":"2024-04-15T15:23:56.261274+0000","last_clean":"2024-04-15T15:23:56.261274+0000","last_became_active":"2024-04-15T15:23:55.270131+0000","last_became_peered":"2024-04-15T15:23:55.270131+0000","last_unstale":"2024-04-15T15:23:56.261274+0000","last_undegraded":"2024-04-15T15:23:56.261274+0000","last_fullsized":"2024-04-15T15:23:56.261274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415461+0000","last_change":"2024-04-15T15:23:55.260659+0000","last_active":"2024-04-15T15:23:56.415461+0000","last_peered":"2024-04-15T15:23:56.415461+0000","last_clean":"2024-04-15T15:23:56.415461+0000","last_became_active":"2024-04-15T15:23:55.258739+0000","last_became_peered":"2024-04-15T15:23:55.258739+0000","last_unstale":"2024-04-15T15:23:56.415461+0000","last_undegraded":"2024-04-15T15:23:56.415461+0000","last_fullsized":"2024-04-15T15:23:56.415461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262634+0000","last_change":"2024-04-15T15:23:55.272818+0000","last_active":"2024-04-15T15:23:56.262634+0000","last_peered":"2024-04-15T15:23:56.262634+0000","last_clean":"2024-04-15T15:23:56.262634+0000","last_became_active":"2024-04-15T15:23:55.269858+0000","last_became_peered":"2024-04-15T15:23:55.269858+0000","last_unstale":"2024-04-15T15:23:56.262634+0000","last_undegraded":"2024-04-15T15:23:56.262634+0000","last_fullsized":"2024-04-15T15:23:56.262634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255726+0000","last_change":"2024-04-15T15:23:55.261128+0000","last_active":"2024-04-15T15:23:56.255726+0000","last_peered":"2024-04-15T15:23:56.255726+0000","last_clean":"2024-04-15T15:23:56.255726+0000","last_became_active":"2024-04-15T15:23:55.259248+0000","last_became_peered":"2024-04-15T15:23:55.259248+0000","last_unstale":"2024-04-15T15:23:56.255726+0000","last_undegraded":"2024-04-15T15:23:56.255726+0000","last_fullsized":"2024-04-15T15:23:56.255726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415563+0000","last_change":"2024-04-15T15:23:55.264402+0000","last_active":"2024-04-15T15:23:56.415563+0000","last_peered":"2024-04-15T15:23:56.415563+0000","last_clean":"2024-04-15T15:23:56.415563+0000","last_became_active":"2024-04-15T15:23:55.264083+0000","last_became_peered":"2024-04-15T15:23:55.264083+0000","last_unstale":"2024-04-15T15:23:56.415563+0000","last_undegraded":"2024-04-15T15:23:56.415563+0000","last_fullsized":"2024-04-15T15:23:56.415563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417369+0000","last_change":"2024-04-15T15:23:55.262147+0000","last_active":"2024-04-15T15:23:56.417369+0000","last_peered":"2024-04-15T15:23:56.417369+0000","last_clean":"2024-04-15T15:23:56.417369+0000","last_became_active":"2024-04-15T15:23:55.261524+0000","last_became_peered":"2024-04-15T15:23:55.261524+0000","last_unstale":"2024-04-15T15:23:56.417369+0000","last_undegraded":"2024-04-15T15:23:56.417369+0000","last_fullsized":"2024-04-15T15:23:56.417369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262811+0000","last_change":"2024-04-15T15:23:55.267385+0000","last_active":"2024-04-15T15:23:56.262811+0000","last_peered":"2024-04-15T15:23:56.262811+0000","last_clean":"2024-04-15T15:23:56.262811+0000","last_became_active":"2024-04-15T15:23:55.267045+0000","last_became_peered":"2024-04-15T15:23:55.267045+0000","last_unstale":"2024-04-15T15:23:56.262811+0000","last_undegraded":"2024-04-15T15:23:56.262811+0000","last_fullsized":"2024-04-15T15:23:56.262811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:35.342060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417143+0000","last_change":"2024-04-15T15:23:55.261039+0000","last_active":"2024-04-15T15:23:56.417143+0000","last_peered":"2024-04-15T15:23:56.417143+0000","last_clean":"2024-04-15T15:23:56.417143+0000","last_became_active":"2024-04-15T15:23:55.260620+0000","last_became_peered":"2024-04-15T15:23:55.260620+0000","last_unstale":"2024-04-15T15:23:56.417143+0000","last_undegraded":"2024-04-15T15:23:56.417143+0000","last_fullsized":"2024-04-15T15:23:56.417143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:27:44.245014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261366+0000","last_change":"2024-04-15T15:23:55.271261+0000","last_active":"2024-04-15T15:23:56.261366+0000","last_peered":"2024-04-15T15:23:56.261366+0000","last_clean":"2024-04-15T15:23:56.261366+0000","last_became_active":"2024-04-15T15:23:55.270954+0000","last_became_peered":"2024-04-15T15:23:55.270954+0000","last_unstale":"2024-04-15T15:23:56.261366+0000","last_undegraded":"2024-04-15T15:23:56.261366+0000","last_fullsized":"2024-04-15T15:23:56.261366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417469+0000","last_change":"2024-04-15T15:23:55.263715+0000","last_active":"2024-04-15T15:23:56.417469+0000","last_peered":"2024-04-15T15:23:56.417469+0000","last_clean":"2024-04-15T15:23:56.417469+0000","last_became_active":"2024-04-15T15:23:55.256992+0000","last_became_peered":"2024-04-15T15:23:55.256992+0000","last_unstale":"2024-04-15T15:23:56.417469+0000","last_undegraded":"2024-04-15T15:23:56.417469+0000","last_fullsized":"2024-04-15T15:23:56.417469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:25.936170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416848+0000","last_change":"2024-04-15T15:23:54.250784+0000","last_active":"2024-04-15T15:23:56.416848+0000","last_peered":"2024-04-15T15:23:56.416848+0000","last_clean":"2024-04-15T15:23:56.416848+0000","last_became_active":"2024-04-15T15:23:54.250347+0000","last_became_peered":"2024-04-15T15:23:54.250347+0000","last_unstale":"2024-04-15T15:23:56.416848+0000","last_undegraded":"2024-04-15T15:23:56.416848+0000","last_fullsized":"2024-04-15T15:23:56.416848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417629+0000","last_change":"2024-04-15T15:23:55.262408+0000","last_active":"2024-04-15T15:23:56.417629+0000","last_peered":"2024-04-15T15:23:56.417629+0000","last_clean":"2024-04-15T15:23:56.417629+0000","last_became_active":"2024-04-15T15:23:55.261815+0000","last_became_peered":"2024-04-15T15:23:55.261815+0000","last_unstale":"2024-04-15T15:23:56.417629+0000","last_undegraded":"2024-04-15T15:23:56.417629+0000","last_fullsized":"2024-04-15T15:23:56.417629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261683+0000","last_change":"2024-04-15T15:23:54.239676+0000","last_active":"2024-04-15T15:23:56.261683+0000","last_peered":"2024-04-15T15:23:56.261683+0000","last_clean":"2024-04-15T15:23:56.261683+0000","last_became_active":"2024-04-15T15:23:54.239084+0000","last_became_peered":"2024-04-15T15:23:54.239084+0000","last_unstale":"2024-04-15T15:23:56.261683+0000","last_undegraded":"2024-04-15T15:23:56.261683+0000","last_fullsized":"2024-04-15T15:23:56.261683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415034+0000","last_change":"2024-04-15T15:23:54.241378+0000","last_active":"2024-04-15T15:23:56.415034+0000","last_peered":"2024-04-15T15:23:56.415034+0000","last_clean":"2024-04-15T15:23:56.415034+0000","last_became_active":"2024-04-15T15:23:54.241084+0000","last_became_peered":"2024-04-15T15:23:54.241084+0000","last_unstale":"2024-04-15T15:23:56.415034+0000","last_undegraded":"2024-04-15T15:23:56.415034+0000","last_fullsized":"2024-04-15T15:23:56.415034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:38:14.692145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.414885+0000","last_change":"2024-04-15T15:23:55.257791+0000","last_active":"2024-04-15T15:23:56.414885+0000","last_peered":"2024-04-15T15:23:56.414885+0000","last_clean":"2024-04-15T15:23:56.414885+0000","last_became_active":"2024-04-15T15:23:55.257531+0000","last_became_peered":"2024-04-15T15:23:55.257531+0000","last_unstale":"2024-04-15T15:23:56.414885+0000","last_undegraded":"2024-04-15T15:23:56.414885+0000","last_fullsized":"2024-04-15T15:23:56.414885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261118+0000","last_change":"2024-04-15T15:23:54.235199+0000","last_active":"2024-04-15T15:23:56.261118+0000","last_peered":"2024-04-15T15:23:56.261118+0000","last_clean":"2024-04-15T15:23:56.261118+0000","last_became_active":"2024-04-15T15:23:54.233620+0000","last_became_peered":"2024-04-15T15:23:54.233620+0000","last_unstale":"2024-04-15T15:23:56.261118+0000","last_undegraded":"2024-04-15T15:23:56.261118+0000","last_fullsized":"2024-04-15T15:23:56.261118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255313+0000","last_change":"2024-04-15T15:23:55.261632+0000","last_active":"2024-04-15T15:23:56.255313+0000","last_peered":"2024-04-15T15:23:56.255313+0000","last_clean":"2024-04-15T15:23:56.255313+0000","last_became_active":"2024-04-15T15:23:55.261351+0000","last_became_peered":"2024-04-15T15:23:55.261351+0000","last_unstale":"2024-04-15T15:23:56.255313+0000","last_undegraded":"2024-04-15T15:23:56.255313+0000","last_fullsized":"2024-04-15T15:23:56.255313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263032+0000","last_change":"2024-04-15T15:23:54.237429+0000","last_active":"2024-04-15T15:23:56.263032+0000","last_peered":"2024-04-15T15:23:56.263032+0000","last_clean":"2024-04-15T15:23:56.263032+0000","last_became_active":"2024-04-15T15:23:54.237045+0000","last_became_peered":"2024-04-15T15:23:54.237045+0000","last_unstale":"2024-04-15T15:23:56.263032+0000","last_undegraded":"2024-04-15T15:23:56.263032+0000","last_fullsized":"2024-04-15T15:23:56.263032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262636+0000","last_change":"2024-04-15T15:23:55.263292+0000","last_active":"2024-04-15T15:23:56.262636+0000","last_peered":"2024-04-15T15:23:56.262636+0000","last_clean":"2024-04-15T15:23:56.262636+0000","last_became_active":"2024-04-15T15:23:55.261899+0000","last_became_peered":"2024-04-15T15:23:55.261899+0000","last_unstale":"2024-04-15T15:23:56.262636+0000","last_undegraded":"2024-04-15T15:23:56.262636+0000","last_fullsized":"2024-04-15T15:23:56.262636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:40.716704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262724+0000","last_change":"2024-04-15T15:23:54.242528+0000","last_active":"2024-04-15T15:23:56.262724+0000","last_peered":"2024-04-15T15:23:56.262724+0000","last_clean":"2024-04-15T15:23:56.262724+0000","last_became_active":"2024-04-15T15:23:54.242233+0000","last_became_peered":"2024-04-15T15:23:54.242233+0000","last_unstale":"2024-04-15T15:23:56.262724+0000","last_undegraded":"2024-04-15T15:23:56.262724+0000","last_fullsized":"2024-04-15T15:23:56.262724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417168+0000","last_change":"2024-04-15T15:23:55.268964+0000","last_active":"2024-04-15T15:23:56.417168+0000","last_peered":"2024-04-15T15:23:56.417168+0000","last_clean":"2024-04-15T15:23:56.417168+0000","last_became_active":"2024-04-15T15:23:55.268348+0000","last_became_peered":"2024-04-15T15:23:55.268348+0000","last_unstale":"2024-04-15T15:23:56.417168+0000","last_undegraded":"2024-04-15T15:23:56.417168+0000","last_fullsized":"2024-04-15T15:23:56.417168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255623+0000","last_change":"2024-04-15T15:23:54.238578+0000","last_active":"2024-04-15T15:23:56.255623+0000","last_peered":"2024-04-15T15:23:56.255623+0000","last_clean":"2024-04-15T15:23:56.255623+0000","last_became_active":"2024-04-15T15:23:54.238332+0000","last_became_peered":"2024-04-15T15:23:54.238332+0000","last_unstale":"2024-04-15T15:23:56.255623+0000","last_undegraded":"2024-04-15T15:23:56.255623+0000","last_fullsized":"2024-04-15T15:23:56.255623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418641+0000","last_change":"2024-04-15T15:23:55.266173+0000","last_active":"2024-04-15T15:23:56.418641+0000","last_peered":"2024-04-15T15:23:56.418641+0000","last_clean":"2024-04-15T15:23:56.418641+0000","last_became_active":"2024-04-15T15:23:55.265877+0000","last_became_peered":"2024-04-15T15:23:55.265877+0000","last_unstale":"2024-04-15T15:23:56.418641+0000","last_undegraded":"2024-04-15T15:23:56.418641+0000","last_fullsized":"2024-04-15T15:23:56.418641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:52.386885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262108+0000","last_change":"2024-04-15T15:23:54.240249+0000","last_active":"2024-04-15T15:23:56.262108+0000","last_peered":"2024-04-15T15:23:56.262108+0000","last_clean":"2024-04-15T15:23:56.262108+0000","last_became_active":"2024-04-15T15:23:54.239151+0000","last_became_peered":"2024-04-15T15:23:54.239151+0000","last_unstale":"2024-04-15T15:23:56.262108+0000","last_undegraded":"2024-04-15T15:23:56.262108+0000","last_fullsized":"2024-04-15T15:23:56.262108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417969+0000","last_change":"2024-04-15T15:23:55.266293+0000","last_active":"2024-04-15T15:23:56.417969+0000","last_peered":"2024-04-15T15:23:56.417969+0000","last_clean":"2024-04-15T15:23:56.417969+0000","last_became_active":"2024-04-15T15:23:55.265962+0000","last_became_peered":"2024-04-15T15:23:55.265962+0000","last_unstale":"2024-04-15T15:23:56.417969+0000","last_undegraded":"2024-04-15T15:23:56.417969+0000","last_fullsized":"2024-04-15T15:23:56.417969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:40.334427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417723+0000","last_change":"2024-04-15T15:23:54.233459+0000","last_active":"2024-04-15T15:23:56.417723+0000","last_peered":"2024-04-15T15:23:56.417723+0000","last_clean":"2024-04-15T15:23:56.417723+0000","last_became_active":"2024-04-15T15:23:54.233056+0000","last_became_peered":"2024-04-15T15:23:54.233056+0000","last_unstale":"2024-04-15T15:23:56.417723+0000","last_undegraded":"2024-04-15T15:23:56.417723+0000","last_fullsized":"2024-04-15T15:23:56.417723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:38:14.692145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417725+0000","last_change":"2024-04-15T15:23:55.261166+0000","last_active":"2024-04-15T15:23:56.417725+0000","last_peered":"2024-04-15T15:23:56.417725+0000","last_clean":"2024-04-15T15:23:56.417725+0000","last_became_active":"2024-04-15T15:23:55.260827+0000","last_became_peered":"2024-04-15T15:23:55.260827+0000","last_unstale":"2024-04-15T15:23:56.417725+0000","last_undegraded":"2024-04-15T15:23:56.417725+0000","last_fullsized":"2024-04-15T15:23:56.417725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262944+0000","last_change":"2024-04-15T15:23:54.233681+0000","last_active":"2024-04-15T15:23:56.262944+0000","last_peered":"2024-04-15T15:23:56.262944+0000","last_clean":"2024-04-15T15:23:56.262944+0000","last_became_active":"2024-04-15T15:23:54.233376+0000","last_became_peered":"2024-04-15T15:23:54.233376+0000","last_unstale":"2024-04-15T15:23:56.262944+0000","last_undegraded":"2024-04-15T15:23:56.262944+0000","last_fullsized":"2024-04-15T15:23:56.262944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419714+0000","last_change":"2024-04-15T15:23:55.275827+0000","last_active":"2024-04-15T15:23:56.419714+0000","last_peered":"2024-04-15T15:23:56.419714+0000","last_clean":"2024-04-15T15:23:56.419714+0000","last_became_active":"2024-04-15T15:23:55.275511+0000","last_became_peered":"2024-04-15T15:23:55.275511+0000","last_unstale":"2024-04-15T15:23:56.419714+0000","last_undegraded":"2024-04-15T15:23:56.419714+0000","last_fullsized":"2024-04-15T15:23:56.419714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:17:08.438022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261296+0000","last_change":"2024-04-15T15:23:54.237666+0000","last_active":"2024-04-15T15:23:56.261296+0000","last_peered":"2024-04-15T15:23:56.261296+0000","last_clean":"2024-04-15T15:23:56.261296+0000","last_became_active":"2024-04-15T15:23:54.237243+0000","last_became_peered":"2024-04-15T15:23:54.237243+0000","last_unstale":"2024-04-15T15:23:56.261296+0000","last_undegraded":"2024-04-15T15:23:56.261296+0000","last_fullsized":"2024-04-15T15:23:56.261296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262905+0000","last_change":"2024-04-15T15:23:55.264795+0000","last_active":"2024-04-15T15:23:56.262905+0000","last_peered":"2024-04-15T15:23:56.262905+0000","last_clean":"2024-04-15T15:23:56.262905+0000","last_became_active":"2024-04-15T15:23:55.264509+0000","last_became_peered":"2024-04-15T15:23:55.264509+0000","last_unstale":"2024-04-15T15:23:56.262905+0000","last_undegraded":"2024-04-15T15:23:56.262905+0000","last_fullsized":"2024-04-15T15:23:56.262905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416505+0000","last_change":"2024-04-15T15:23:54.233260+0000","last_active":"2024-04-15T15:23:56.416505+0000","last_peered":"2024-04-15T15:23:56.416505+0000","last_clean":"2024-04-15T15:23:56.416505+0000","last_became_active":"2024-04-15T15:23:54.231670+0000","last_became_peered":"2024-04-15T15:23:54.231670+0000","last_unstale":"2024-04-15T15:23:56.416505+0000","last_undegraded":"2024-04-15T15:23:56.416505+0000","last_fullsized":"2024-04-15T15:23:56.416505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:33:24.512549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418111+0000","last_change":"2024-04-15T15:23:55.278880+0000","last_active":"2024-04-15T15:23:56.418111+0000","last_peered":"2024-04-15T15:23:56.418111+0000","last_clean":"2024-04-15T15:23:56.418111+0000","last_became_active":"2024-04-15T15:23:55.278617+0000","last_became_peered":"2024-04-15T15:23:55.278617+0000","last_unstale":"2024-04-15T15:23:56.418111+0000","last_undegraded":"2024-04-15T15:23:56.418111+0000","last_fullsized":"2024-04-15T15:23:56.418111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:16:50.418929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418140+0000","last_change":"2024-04-15T15:23:54.239459+0000","last_active":"2024-04-15T15:23:56.418140+0000","last_peered":"2024-04-15T15:23:56.418140+0000","last_clean":"2024-04-15T15:23:56.418140+0000","last_became_active":"2024-04-15T15:23:54.239033+0000","last_became_peered":"2024-04-15T15:23:54.239033+0000","last_unstale":"2024-04-15T15:23:56.418140+0000","last_undegraded":"2024-04-15T15:23:56.418140+0000","last_fullsized":"2024-04-15T15:23:56.418140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:16.311793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419071+0000","last_change":"2024-04-15T15:23:55.277191+0000","last_active":"2024-04-15T15:23:56.419071+0000","last_peered":"2024-04-15T15:23:56.419071+0000","last_clean":"2024-04-15T15:23:56.419071+0000","last_became_active":"2024-04-15T15:23:55.276803+0000","last_became_peered":"2024-04-15T15:23:55.276803+0000","last_unstale":"2024-04-15T15:23:56.419071+0000","last_undegraded":"2024-04-15T15:23:56.419071+0000","last_fullsized":"2024-04-15T15:23:56.419071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:53:48.793142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417566+0000","last_change":"2024-04-15T15:23:54.230441+0000","last_active":"2024-04-15T15:23:56.417566+0000","last_peered":"2024-04-15T15:23:56.417566+0000","last_clean":"2024-04-15T15:23:56.417566+0000","last_became_active":"2024-04-15T15:23:54.230051+0000","last_became_peered":"2024-04-15T15:23:54.230051+0000","last_unstale":"2024-04-15T15:23:56.417566+0000","last_undegraded":"2024-04-15T15:23:56.417566+0000","last_fullsized":"2024-04-15T15:23:56.417566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:29.938678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418932+0000","last_change":"2024-04-15T15:23:55.281762+0000","last_active":"2024-04-15T15:23:56.418932+0000","last_peered":"2024-04-15T15:23:56.418932+0000","last_clean":"2024-04-15T15:23:56.418932+0000","last_became_active":"2024-04-15T15:23:55.281494+0000","last_became_peered":"2024-04-15T15:23:55.281494+0000","last_unstale":"2024-04-15T15:23:56.418932+0000","last_undegraded":"2024-04-15T15:23:56.418932+0000","last_fullsized":"2024-04-15T15:23:56.418932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:42:09.480734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255780+0000","last_change":"2024-04-15T15:23:54.234786+0000","last_active":"2024-04-15T15:23:56.255780+0000","last_peered":"2024-04-15T15:23:56.255780+0000","last_clean":"2024-04-15T15:23:56.255780+0000","last_became_active":"2024-04-15T15:23:54.234484+0000","last_became_peered":"2024-04-15T15:23:54.234484+0000","last_unstale":"2024-04-15T15:23:56.255780+0000","last_undegraded":"2024-04-15T15:23:56.255780+0000","last_fullsized":"2024-04-15T15:23:56.255780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:05.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262958+0000","last_change":"2024-04-15T15:23:55.267889+0000","last_active":"2024-04-15T15:23:56.262958+0000","last_peered":"2024-04-15T15:23:56.262958+0000","last_clean":"2024-04-15T15:23:56.262958+0000","last_became_active":"2024-04-15T15:23:55.267209+0000","last_became_peered":"2024-04-15T15:23:55.267209+0000","last_unstale":"2024-04-15T15:23:56.262958+0000","last_undegraded":"2024-04-15T15:23:56.262958+0000","last_fullsized":"2024-04-15T15:23:56.262958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:40.334427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416588+0000","last_change":"2024-04-15T15:23:54.237963+0000","last_active":"2024-04-15T15:23:56.416588+0000","last_peered":"2024-04-15T15:23:56.416588+0000","last_clean":"2024-04-15T15:23:56.416588+0000","last_became_active":"2024-04-15T15:23:54.237726+0000","last_became_peered":"2024-04-15T15:23:54.237726+0000","last_unstale":"2024-04-15T15:23:56.416588+0000","last_undegraded":"2024-04-15T15:23:56.416588+0000","last_fullsized":"2024-04-15T15:23:56.416588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:45:06.635874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418333+0000","last_change":"2024-04-15T15:23:55.272911+0000","last_active":"2024-04-15T15:23:56.418333+0000","last_peered":"2024-04-15T15:23:56.418333+0000","last_clean":"2024-04-15T15:23:56.418333+0000","last_became_active":"2024-04-15T15:23:55.272642+0000","last_became_peered":"2024-04-15T15:23:55.272642+0000","last_unstale":"2024-04-15T15:23:56.418333+0000","last_undegraded":"2024-04-15T15:23:56.418333+0000","last_fullsized":"2024-04-15T15:23:56.418333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:27.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417306+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:23:56.417306+0000","last_peered":"2024-04-15T15:23:56.417306+0000","last_clean":"2024-04-15T15:23:56.417306+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:23:56.417306+0000","last_undegraded":"2024-04-15T15:23:56.417306+0000","last_fullsized":"2024-04-15T15:23:56.417306+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261029+0000","last_change":"2024-04-15T15:23:54.232974+0000","last_active":"2024-04-15T15:23:56.261029+0000","last_peered":"2024-04-15T15:23:56.261029+0000","last_clean":"2024-04-15T15:23:56.261029+0000","last_became_active":"2024-04-15T15:23:54.230808+0000","last_became_peered":"2024-04-15T15:23:54.230808+0000","last_unstale":"2024-04-15T15:23:56.261029+0000","last_undegraded":"2024-04-15T15:23:56.261029+0000","last_fullsized":"2024-04-15T15:23:56.261029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:46:30.058785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418655+0000","last_change":"2024-04-15T15:23:55.269058+0000","last_active":"2024-04-15T15:23:56.418655+0000","last_peered":"2024-04-15T15:23:56.418655+0000","last_clean":"2024-04-15T15:23:56.418655+0000","last_became_active":"2024-04-15T15:23:55.268267+0000","last_became_peered":"2024-04-15T15:23:55.268267+0000","last_unstale":"2024-04-15T15:23:56.418655+0000","last_undegraded":"2024-04-15T15:23:56.418655+0000","last_fullsized":"2024-04-15T15:23:56.418655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:30:18.931762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417047+0000","last_change":"2024-04-15T15:23:54.251259+0000","last_active":"2024-04-15T15:23:56.417047+0000","last_peered":"2024-04-15T15:23:56.417047+0000","last_clean":"2024-04-15T15:23:56.417047+0000","last_became_active":"2024-04-15T15:23:54.250985+0000","last_became_peered":"2024-04-15T15:23:54.250985+0000","last_unstale":"2024-04-15T15:23:56.417047+0000","last_undegraded":"2024-04-15T15:23:56.417047+0000","last_fullsized":"2024-04-15T15:23:56.417047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:44:11.419953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418428+0000","last_change":"2024-04-15T15:23:55.267279+0000","last_active":"2024-04-15T15:23:56.418428+0000","last_peered":"2024-04-15T15:23:56.418428+0000","last_clean":"2024-04-15T15:23:56.418428+0000","last_became_active":"2024-04-15T15:23:55.266971+0000","last_became_peered":"2024-04-15T15:23:55.266971+0000","last_unstale":"2024-04-15T15:23:56.418428+0000","last_undegraded":"2024-04-15T15:23:56.418428+0000","last_fullsized":"2024-04-15T15:23:56.418428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417318+0000","last_change":"2024-04-15T15:23:54.234761+0000","last_active":"2024-04-15T15:23:56.417318+0000","last_peered":"2024-04-15T15:23:56.417318+0000","last_clean":"2024-04-15T15:23:56.417318+0000","last_became_active":"2024-04-15T15:23:54.234467+0000","last_became_peered":"2024-04-15T15:23:54.234467+0000","last_unstale":"2024-04-15T15:23:56.417318+0000","last_undegraded":"2024-04-15T15:23:56.417318+0000","last_fullsized":"2024-04-15T15:23:56.417318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:47:42.194686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255347+0000","last_change":"2024-04-15T15:23:55.244630+0000","last_active":"2024-04-15T15:23:56.255347+0000","last_peered":"2024-04-15T15:23:56.255347+0000","last_clean":"2024-04-15T15:23:56.255347+0000","last_became_active":"2024-04-15T15:23:55.244208+0000","last_became_peered":"2024-04-15T15:23:55.244208+0000","last_unstale":"2024-04-15T15:23:56.255347+0000","last_undegraded":"2024-04-15T15:23:56.255347+0000","last_fullsized":"2024-04-15T15:23:56.255347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263182+0000","last_change":"2024-04-15T15:23:54.225774+0000","last_active":"2024-04-15T15:23:56.263182+0000","last_peered":"2024-04-15T15:23:56.263182+0000","last_clean":"2024-04-15T15:23:56.263182+0000","last_became_active":"2024-04-15T15:23:54.223951+0000","last_became_peered":"2024-04-15T15:23:54.223951+0000","last_unstale":"2024-04-15T15:23:56.263182+0000","last_undegraded":"2024-04-15T15:23:56.263182+0000","last_fullsized":"2024-04-15T15:23:56.263182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417834+0000","last_change":"2024-04-15T15:23:55.261370+0000","last_active":"2024-04-15T15:23:56.417834+0000","last_peered":"2024-04-15T15:23:56.417834+0000","last_clean":"2024-04-15T15:23:56.417834+0000","last_became_active":"2024-04-15T15:23:55.260930+0000","last_became_peered":"2024-04-15T15:23:55.260930+0000","last_unstale":"2024-04-15T15:23:56.417834+0000","last_undegraded":"2024-04-15T15:23:56.417834+0000","last_fullsized":"2024-04-15T15:23:56.417834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:54.270601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255534+0000","last_change":"2024-04-15T15:23:54.223225+0000","last_active":"2024-04-15T15:23:56.255534+0000","last_peered":"2024-04-15T15:23:56.255534+0000","last_clean":"2024-04-15T15:23:56.255534+0000","last_became_active":"2024-04-15T15:23:54.222903+0000","last_became_peered":"2024-04-15T15:23:54.222903+0000","last_unstale":"2024-04-15T15:23:56.255534+0000","last_undegraded":"2024-04-15T15:23:56.255534+0000","last_fullsized":"2024-04-15T15:23:56.255534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:51.381976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.264575+0000","last_change":"2024-04-15T15:23:55.270383+0000","last_active":"2024-04-15T15:23:56.264575+0000","last_peered":"2024-04-15T15:23:56.264575+0000","last_clean":"2024-04-15T15:23:56.264575+0000","last_became_active":"2024-04-15T15:23:55.267157+0000","last_became_peered":"2024-04-15T15:23:55.267157+0000","last_unstale":"2024-04-15T15:23:56.264575+0000","last_undegraded":"2024-04-15T15:23:56.264575+0000","last_fullsized":"2024-04-15T15:23:56.264575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:27.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419141+0000","last_change":"2024-04-15T15:23:54.251497+0000","last_active":"2024-04-15T15:23:56.419141+0000","last_peered":"2024-04-15T15:23:56.419141+0000","last_clean":"2024-04-15T15:23:56.419141+0000","last_became_active":"2024-04-15T15:23:54.250660+0000","last_became_peered":"2024-04-15T15:23:54.250660+0000","last_unstale":"2024-04-15T15:23:56.419141+0000","last_undegraded":"2024-04-15T15:23:56.419141+0000","last_fullsized":"2024-04-15T15:23:56.419141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:14.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255397+0000","last_change":"2024-04-15T15:23:55.249702+0000","last_active":"2024-04-15T15:23:56.255397+0000","last_peered":"2024-04-15T15:23:56.255397+0000","last_clean":"2024-04-15T15:23:56.255397+0000","last_became_active":"2024-04-15T15:23:55.249084+0000","last_became_peered":"2024-04-15T15:23:55.249084+0000","last_unstale":"2024-04-15T15:23:56.255397+0000","last_undegraded":"2024-04-15T15:23:56.255397+0000","last_fullsized":"2024-04-15T15:23:56.255397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255845+0000","last_change":"2024-04-15T15:23:54.241520+0000","last_active":"2024-04-15T15:23:56.255845+0000","last_peered":"2024-04-15T15:23:56.255845+0000","last_clean":"2024-04-15T15:23:56.255845+0000","last_became_active":"2024-04-15T15:23:54.241103+0000","last_became_peered":"2024-04-15T15:23:54.241103+0000","last_unstale":"2024-04-15T15:23:56.255845+0000","last_undegraded":"2024-04-15T15:23:56.255845+0000","last_fullsized":"2024-04-15T15:23:56.255845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:44:11.419953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415819+0000","last_change":"2024-04-15T15:23:55.265567+0000","last_active":"2024-04-15T15:23:56.415819+0000","last_peered":"2024-04-15T15:23:56.415819+0000","last_clean":"2024-04-15T15:23:56.415819+0000","last_became_active":"2024-04-15T15:23:55.262905+0000","last_became_peered":"2024-04-15T15:23:55.262905+0000","last_unstale":"2024-04-15T15:23:56.415819+0000","last_undegraded":"2024-04-15T15:23:56.415819+0000","last_fullsized":"2024-04-15T15:23:56.415819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263461+0000","last_change":"2024-04-15T15:23:54.252083+0000","last_active":"2024-04-15T15:23:56.263461+0000","last_peered":"2024-04-15T15:23:56.263461+0000","last_clean":"2024-04-15T15:23:56.263461+0000","last_became_active":"2024-04-15T15:23:54.251796+0000","last_became_peered":"2024-04-15T15:23:54.251796+0000","last_unstale":"2024-04-15T15:23:56.263461+0000","last_undegraded":"2024-04-15T15:23:56.263461+0000","last_fullsized":"2024-04-15T15:23:56.263461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:46:30.058785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416720+0000","last_change":"2024-04-15T15:23:55.257995+0000","last_active":"2024-04-15T15:23:56.416720+0000","last_peered":"2024-04-15T15:23:56.416720+0000","last_clean":"2024-04-15T15:23:56.416720+0000","last_became_active":"2024-04-15T15:23:55.257718+0000","last_became_peered":"2024-04-15T15:23:55.257718+0000","last_unstale":"2024-04-15T15:23:56.416720+0000","last_undegraded":"2024-04-15T15:23:56.416720+0000","last_fullsized":"2024-04-15T15:23:56.416720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:08:29.731069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416748+0000","last_change":"2024-04-15T15:23:54.250977+0000","last_active":"2024-04-15T15:23:56.416748+0000","last_peered":"2024-04-15T15:23:56.416748+0000","last_clean":"2024-04-15T15:23:56.416748+0000","last_became_active":"2024-04-15T15:23:54.250302+0000","last_became_peered":"2024-04-15T15:23:54.250302+0000","last_unstale":"2024-04-15T15:23:56.416748+0000","last_undegraded":"2024-04-15T15:23:56.416748+0000","last_fullsized":"2024-04-15T15:23:56.416748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:05.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418747+0000","last_change":"2024-04-15T15:23:55.260220+0000","last_active":"2024-04-15T15:23:56.418747+0000","last_peered":"2024-04-15T15:23:56.418747+0000","last_clean":"2024-04-15T15:23:56.418747+0000","last_became_active":"2024-04-15T15:23:55.259361+0000","last_became_peered":"2024-04-15T15:23:55.259361+0000","last_unstale":"2024-04-15T15:23:56.418747+0000","last_undegraded":"2024-04-15T15:23:56.418747+0000","last_fullsized":"2024-04-15T15:23:56.418747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:06.899022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255405+0000","last_change":"2024-04-15T15:23:54.236256+0000","last_active":"2024-04-15T15:23:56.255405+0000","last_peered":"2024-04-15T15:23:56.255405+0000","last_clean":"2024-04-15T15:23:56.255405+0000","last_became_active":"2024-04-15T15:23:54.235919+0000","last_became_peered":"2024-04-15T15:23:54.235919+0000","last_unstale":"2024-04-15T15:23:56.255405+0000","last_undegraded":"2024-04-15T15:23:56.255405+0000","last_fullsized":"2024-04-15T15:23:56.255405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415070+0000","last_change":"2024-04-15T15:23:55.258682+0000","last_active":"2024-04-15T15:23:56.415070+0000","last_peered":"2024-04-15T15:23:56.415070+0000","last_clean":"2024-04-15T15:23:56.415070+0000","last_became_active":"2024-04-15T15:23:55.258183+0000","last_became_peered":"2024-04-15T15:23:55.258183+0000","last_unstale":"2024-04-15T15:23:56.415070+0000","last_undegraded":"2024-04-15T15:23:56.415070+0000","last_fullsized":"2024-04-15T15:23:56.415070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255301+0000","last_change":"2024-04-15T15:23:54.233682+0000","last_active":"2024-04-15T15:23:56.255301+0000","last_peered":"2024-04-15T15:23:56.255301+0000","last_clean":"2024-04-15T15:23:56.255301+0000","last_became_active":"2024-04-15T15:23:54.233373+0000","last_became_peered":"2024-04-15T15:23:54.233373+0000","last_unstale":"2024-04-15T15:23:56.255301+0000","last_undegraded":"2024-04-15T15:23:56.255301+0000","last_fullsized":"2024-04-15T15:23:56.255301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416048+0000","last_change":"2024-04-15T15:23:55.272358+0000","last_active":"2024-04-15T15:23:56.416048+0000","last_peered":"2024-04-15T15:23:56.416048+0000","last_clean":"2024-04-15T15:23:56.416048+0000","last_became_active":"2024-04-15T15:23:55.271660+0000","last_became_peered":"2024-04-15T15:23:55.271660+0000","last_unstale":"2024-04-15T15:23:56.416048+0000","last_undegraded":"2024-04-15T15:23:56.416048+0000","last_fullsized":"2024-04-15T15:23:56.416048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419745+0000","last_change":"2024-04-15T15:23:54.251693+0000","last_active":"2024-04-15T15:23:56.419745+0000","last_peered":"2024-04-15T15:23:56.419745+0000","last_clean":"2024-04-15T15:23:56.419745+0000","last_became_active":"2024-04-15T15:23:54.250255+0000","last_became_peered":"2024-04-15T15:23:54.250255+0000","last_unstale":"2024-04-15T15:23:56.419745+0000","last_undegraded":"2024-04-15T15:23:56.419745+0000","last_fullsized":"2024-04-15T15:23:56.419745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:08:28.726161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261853+0000","last_change":"2024-04-15T15:23:55.278856+0000","last_active":"2024-04-15T15:23:56.261853+0000","last_peered":"2024-04-15T15:23:56.261853+0000","last_clean":"2024-04-15T15:23:56.261853+0000","last_became_active":"2024-04-15T15:23:55.278605+0000","last_became_peered":"2024-04-15T15:23:55.278605+0000","last_unstale":"2024-04-15T15:23:56.261853+0000","last_undegraded":"2024-04-15T15:23:56.261853+0000","last_fullsized":"2024-04-15T15:23:56.261853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.264390+0000","last_change":"2024-04-15T15:23:55.260285+0000","last_active":"2024-04-15T15:23:56.264390+0000","last_peered":"2024-04-15T15:23:56.264390+0000","last_clean":"2024-04-15T15:23:56.264390+0000","last_became_active":"2024-04-15T15:23:55.260016+0000","last_became_peered":"2024-04-15T15:23:55.260016+0000","last_unstale":"2024-04-15T15:23:56.264390+0000","last_undegraded":"2024-04-15T15:23:56.264390+0000","last_fullsized":"2024-04-15T15:23:56.264390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:58.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418779+0000","last_change":"2024-04-15T15:23:54.250621+0000","last_active":"2024-04-15T15:23:56.418779+0000","last_peered":"2024-04-15T15:23:56.418779+0000","last_clean":"2024-04-15T15:23:56.418779+0000","last_became_active":"2024-04-15T15:23:54.250204+0000","last_became_peered":"2024-04-15T15:23:54.250204+0000","last_unstale":"2024-04-15T15:23:56.418779+0000","last_undegraded":"2024-04-15T15:23:56.418779+0000","last_fullsized":"2024-04-15T15:23:56.418779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:54:51.507224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262544+0000","last_change":"2024-04-15T15:23:55.263054+0000","last_active":"2024-04-15T15:23:56.262544+0000","last_peered":"2024-04-15T15:23:56.262544+0000","last_clean":"2024-04-15T15:23:56.262544+0000","last_became_active":"2024-04-15T15:23:55.262742+0000","last_became_peered":"2024-04-15T15:23:55.262742+0000","last_unstale":"2024-04-15T15:23:56.262544+0000","last_undegraded":"2024-04-15T15:23:56.262544+0000","last_fullsized":"2024-04-15T15:23:56.262544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:30:18.931762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418457+0000","last_change":"2024-04-15T15:23:54.250572+0000","last_active":"2024-04-15T15:23:56.418457+0000","last_peered":"2024-04-15T15:23:56.418457+0000","last_clean":"2024-04-15T15:23:56.418457+0000","last_became_active":"2024-04-15T15:23:54.250229+0000","last_became_peered":"2024-04-15T15:23:54.250229+0000","last_unstale":"2024-04-15T15:23:56.418457+0000","last_undegraded":"2024-04-15T15:23:56.418457+0000","last_fullsized":"2024-04-15T15:23:56.418457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:15.052794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416985+0000","last_change":"2024-04-15T15:23:55.269263+0000","last_active":"2024-04-15T15:23:56.416985+0000","last_peered":"2024-04-15T15:23:56.416985+0000","last_clean":"2024-04-15T15:23:56.416985+0000","last_became_active":"2024-04-15T15:23:55.268706+0000","last_became_peered":"2024-04-15T15:23:55.268706+0000","last_unstale":"2024-04-15T15:23:56.416985+0000","last_undegraded":"2024-04-15T15:23:56.416985+0000","last_fullsized":"2024-04-15T15:23:56.416985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.057703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415538+0000","last_change":"2024-04-15T15:23:54.229794+0000","last_active":"2024-04-15T15:23:56.415538+0000","last_peered":"2024-04-15T15:23:56.415538+0000","last_clean":"2024-04-15T15:23:56.415538+0000","last_became_active":"2024-04-15T15:23:54.229509+0000","last_became_peered":"2024-04-15T15:23:54.229509+0000","last_unstale":"2024-04-15T15:23:56.415538+0000","last_undegraded":"2024-04-15T15:23:56.415538+0000","last_fullsized":"2024-04-15T15:23:56.415538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:49:04.203638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415289+0000","last_change":"2024-04-15T15:23:55.274587+0000","last_active":"2024-04-15T15:23:56.415289+0000","last_peered":"2024-04-15T15:23:56.415289+0000","last_clean":"2024-04-15T15:23:56.415289+0000","last_became_active":"2024-04-15T15:23:55.274127+0000","last_became_peered":"2024-04-15T15:23:55.274127+0000","last_unstale":"2024-04-15T15:23:56.415289+0000","last_undegraded":"2024-04-15T15:23:56.415289+0000","last_fullsized":"2024-04-15T15:23:56.415289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417220+0000","last_change":"2024-04-15T15:23:54.236884+0000","last_active":"2024-04-15T15:23:56.417220+0000","last_peered":"2024-04-15T15:23:56.417220+0000","last_clean":"2024-04-15T15:23:56.417220+0000","last_became_active":"2024-04-15T15:23:54.236583+0000","last_became_peered":"2024-04-15T15:23:54.236583+0000","last_unstale":"2024-04-15T15:23:56.417220+0000","last_undegraded":"2024-04-15T15:23:56.417220+0000","last_fullsized":"2024-04-15T15:23:56.417220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:23:59.714 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:23:59.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.961+0000 7fb3e3e00640 1 Processor -- start 2024-04-15T15:23:59.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.961+0000 7fb3e3e00640 1 -- start start 2024-04-15T15:23:59.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.962+0000 7fb3e3e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb3e40cee00 0x7fb3e406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.962+0000 7fb3e3e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb3e415d2f0 0x7fb3e40723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.962+0000 7fb3e3e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e415dbb0 con 0x7fb3e4071f50 2024-04-15T15:23:59.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.962+0000 7fb3e3e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e406f4f0 con 0x7fb3e40cee00 2024-04-15T15:23:59.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.962+0000 7fb3e3e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e40728f0 con 0x7fb3e415d2f0 2024-04-15T15:23:59.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.963+0000 7fb3dbe00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fb3e4071f50 0x7fb3e415cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33390/0 (socket says 172.21.15.150:33390) 2024-04-15T15:23:59.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.963+0000 7fb3dbe00640 1 -- 172.21.15.150:0/456389449 learned_addr learned my addr 172.21.15.150:0/456389449 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:23:59.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.963+0000 7fb3e2a00640 1 --2- 172.21.15.150:0/456389449 >> v2:172.21.15.150:3300/0 conn(0x7fb3e40cee00 0x7fb3e406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.963+0000 7fb3e3400640 1 --2- 172.21.15.150:0/456389449 >> v2:172.21.15.187:3300/0 conn(0x7fb3e415d2f0 0x7fb3e40723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.963+0000 7fb3e2000640 1 -- 172.21.15.150:0/456389449 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325259859 0 0) 0x7fb3e415dbb0 con 0x7fb3e4071f50 2024-04-15T15:23:59.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.963+0000 7fb3e2000640 1 -- 172.21.15.150:0/456389449 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3c8003520 con 0x7fb3e4071f50 2024-04-15T15:23:59.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.964+0000 7fb3e2a00640 1 -- 172.21.15.150:0/456389449 >> v2:172.21.15.187:3300/0 conn(0x7fb3e415d2f0 msgr2=0x7fb3e40723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:23:59.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.964+0000 7fb3e2a00640 1 --2- 172.21.15.150:0/456389449 >> v2:172.21.15.187:3300/0 conn(0x7fb3e415d2f0 0x7fb3e40723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.964+0000 7fb3e2a00640 1 -- 172.21.15.150:0/456389449 >> v1:172.21.15.150:6789/0 conn(0x7fb3e4071f50 legacy=0x7fb3e415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.964+0000 7fb3e2a00640 1 -- 172.21.15.150:0/456389449 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e415d8d0 con 0x7fb3e40cee00 2024-04-15T15:23:59.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.964+0000 7fb3e3400640 1 --2- 172.21.15.150:0/456389449 >> v2:172.21.15.187:3300/0 conn(0x7fb3e415d2f0 0x7fb3e40723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:23:59.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.965+0000 7fb3e2a00640 1 --2- 172.21.15.150:0/456389449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e406e8c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3e415a7b0 tx=0x7fb3d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745d77c577506f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.965+0000 7fb3e2000640 1 -- 172.21.15.150:0/456389449 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4035020 con 0x7fb3e40cee00 2024-04-15T15:23:59.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.965+0000 7fb3e2000640 1 -- 172.21.15.150:0/456389449 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d4031830 con 0x7fb3e40cee00 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.965+0000 7fb3e2000640 1 -- 172.21.15.150:0/456389449 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4031c00 con 0x7fb3e40cee00 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.966+0000 7fb3e3e00640 1 -- 172.21.15.150:0/456389449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 msgr2=0x7fb3e406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.966+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/456389449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e406e8c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3e415a7b0 tx=0x7fb3d402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.966+0000 7fb3e3e00640 1 -- 172.21.15.150:0/456389449 shutdown_connections 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.966+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/456389449 >> v2:172.21.15.187:3300/0 conn(0x7fb3e415d2f0 0x7fb3e40723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.966+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/456389449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e406e8c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.966+0000 7fb3e3e00640 1 -- 172.21.15.150:0/456389449 >> 172.21.15.150:0/456389449 conn(0x7fb3e406b930 msgr2=0x7fb3e406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.966+0000 7fb3e3e00640 1 -- 172.21.15.150:0/456389449 shutdown_connections 2024-04-15T15:23:59.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.966+0000 7fb3e3e00640 1 -- 172.21.15.150:0/456389449 wait complete. 2024-04-15T15:23:59.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.968+0000 7fb3e3e00640 1 Processor -- start 2024-04-15T15:23:59.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.968+0000 7fb3e3e00640 1 -- start start 2024-04-15T15:23:59.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3e3e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4071f50 0x7fb3e4147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3e3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3e3e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e415d2f0 0x7fb3e4156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3e3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e406e3a0 con 0x7fb3e40cee00 2024-04-15T15:23:59.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3e3e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e406e0c0 con 0x7fb3e4071f50 2024-04-15T15:23:59.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3e3e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e415dda0 con 0x7fb3e415d2f0 2024-04-15T15:23:59.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3dbe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3dbe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e415ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44398/0 (socket says 172.21.15.150:44398) 2024-04-15T15:23:59.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.969+0000 7fb3dbe00640 1 -- 172.21.15.150:0/1841705147 learned_addr learned my addr 172.21.15.150:0/1841705147 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:23:59.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3e2a00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4071f50 0x7fb3e4147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3e3400640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e415d2f0 0x7fb3e4156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3dbe00640 1 -- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e415d2f0 msgr2=0x7fb3e4156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3dbe00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e415d2f0 0x7fb3e4156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3dbe00640 1 -- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4071f50 msgr2=0x7fb3e4147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:23:59.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3dbe00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4071f50 0x7fb3e4147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:23:59.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3dbe00640 1 -- 172.21.15.150:0/1841705147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e4157730 con 0x7fb3e40cee00 2024-04-15T15:23:59.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3dbe00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e415ebb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00c460 tx=0x7fb3cc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3daa00640 1 -- 172.21.15.150:0/1841705147 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc017070 con 0x7fb3e40cee00 2024-04-15T15:23:59.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3daa00640 1 -- 172.21.15.150:0/1841705147 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3cc005ce0 con 0x7fb3e40cee00 2024-04-15T15:23:59.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3daa00640 1 -- 172.21.15.150:0/1841705147 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc0072f0 con 0x7fb3e40cee00 2024-04-15T15:23:59.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.970+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e4157a10 con 0x7fb3e40cee00 2024-04-15T15:23:59.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.971+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e415e690 con 0x7fb3e40cee00 2024-04-15T15:23:59.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.972+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e415e890 con 0x7fb3e40cee00 2024-04-15T15:23:59.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.972+0000 7fb3daa00640 1 -- 172.21.15.150:0/1841705147 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3cc01a070 con 0x7fb3e40cee00 2024-04-15T15:23:59.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.973+0000 7fb3daa00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3c006c8f0 0x7fb3c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:23:59.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.973+0000 7fb3daa00640 1 -- 172.21.15.150:0/1841705147 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb3cc08b0c0 con 0x7fb3e40cee00 2024-04-15T15:23:59.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.974+0000 7fb3e2a00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3c006c8f0 0x7fb3c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:23:59.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.975+0000 7fb3e2a00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3c006c8f0 0x7fb3c006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4036000 tx=0x7fb3d403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:23:59.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:23:59.978+0000 7fb3daa00640 1 -- 172.21.15.150:0/1841705147 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3e415e890 con 0x7fb3e40cee00 2024-04-15T15:24:00.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.176+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3e40755c0 con 0x7fb3c006c8f0 2024-04-15T15:24:00.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.195+0000 7fb3daa00640 1 -- 172.21.15.150:0/1841705147 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fb3e40755c0 con 0x7fb3c006c8f0 2024-04-15T15:24:00.199 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:00.200 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3c006c8f0 msgr2=0x7fb3c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3c006c8f0 0x7fb3c006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4036000 tx=0x7fb3d403c040 comp rx=0 tx=0).stop 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 msgr2=0x7fb3e415ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e415ebb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00c460 tx=0x7fb3cc00c920 comp rx=0 tx=0).stop 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 shutdown_connections 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3c006c8f0 0x7fb3c006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3e415d2f0 0x7fb3e4156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e40cee00 0x7fb3e415ebb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 --2- 172.21.15.150:0/1841705147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3e4071f50 0x7fb3e4147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 >> 172.21.15.150:0/1841705147 conn(0x7fb3e406b930 msgr2=0x7fb3e40b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:00.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 shutdown_connections 2024-04-15T15:24:00.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.200+0000 7fb3e3e00640 1 -- 172.21.15.150:0/1841705147 wait complete. 2024-04-15T15:24:00.221 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-15T15:23:59.217450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219148,"kb_used_data":2940,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755572,"statfs":{"total":773094113280,"available":772869705728,"internally_reserved":0,"allocated":3010560,"data_stored":1789528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002052"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418140+0000","last_change":"2024-04-15T15:23:55.263891+0000","last_active":"2024-04-15T15:23:56.418140+0000","last_peered":"2024-04-15T15:23:56.418140+0000","last_clean":"2024-04-15T15:23:56.418140+0000","last_became_active":"2024-04-15T15:23:55.263577+0000","last_became_peered":"2024-04-15T15:23:55.263577+0000","last_unstale":"2024-04-15T15:23:56.418140+0000","last_undegraded":"2024-04-15T15:23:56.418140+0000","last_fullsized":"2024-04-15T15:23:56.418140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:58.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255982+0000","last_change":"2024-04-15T15:23:55.259842+0000","last_active":"2024-04-15T15:23:56.255982+0000","last_peered":"2024-04-15T15:23:56.255982+0000","last_clean":"2024-04-15T15:23:56.255982+0000","last_became_active":"2024-04-15T15:23:55.259375+0000","last_became_peered":"2024-04-15T15:23:55.259375+0000","last_unstale":"2024-04-15T15:23:56.255982+0000","last_undegraded":"2024-04-15T15:23:56.255982+0000","last_fullsized":"2024-04-15T15:23:56.255982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418953+0000","last_change":"2024-04-15T15:23:55.249464+0000","last_active":"2024-04-15T15:23:56.418953+0000","last_peered":"2024-04-15T15:23:56.418953+0000","last_clean":"2024-04-15T15:23:56.418953+0000","last_became_active":"2024-04-15T15:23:55.248697+0000","last_became_peered":"2024-04-15T15:23:55.248697+0000","last_unstale":"2024-04-15T15:23:56.418953+0000","last_undegraded":"2024-04-15T15:23:56.418953+0000","last_fullsized":"2024-04-15T15:23:56.418953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262156+0000","last_change":"2024-04-15T15:23:55.267985+0000","last_active":"2024-04-15T15:23:56.262156+0000","last_peered":"2024-04-15T15:23:56.262156+0000","last_clean":"2024-04-15T15:23:56.262156+0000","last_became_active":"2024-04-15T15:23:55.267575+0000","last_became_peered":"2024-04-15T15:23:55.267575+0000","last_unstale":"2024-04-15T15:23:56.262156+0000","last_undegraded":"2024-04-15T15:23:56.262156+0000","last_fullsized":"2024-04-15T15:23:56.262156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:11.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419665+0000","last_change":"2024-04-15T15:23:55.260235+0000","last_active":"2024-04-15T15:23:56.419665+0000","last_peered":"2024-04-15T15:23:56.419665+0000","last_clean":"2024-04-15T15:23:56.419665+0000","last_became_active":"2024-04-15T15:23:55.259947+0000","last_became_peered":"2024-04-15T15:23:55.259947+0000","last_unstale":"2024-04-15T15:23:56.419665+0000","last_undegraded":"2024-04-15T15:23:56.419665+0000","last_fullsized":"2024-04-15T15:23:56.419665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:18:41.223608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261963+0000","last_change":"2024-04-15T15:23:55.257936+0000","last_active":"2024-04-15T15:23:56.261963+0000","last_peered":"2024-04-15T15:23:56.261963+0000","last_clean":"2024-04-15T15:23:56.261963+0000","last_became_active":"2024-04-15T15:23:55.257397+0000","last_became_peered":"2024-04-15T15:23:55.257397+0000","last_unstale":"2024-04-15T15:23:56.261963+0000","last_undegraded":"2024-04-15T15:23:56.261963+0000","last_fullsized":"2024-04-15T15:23:56.261963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417763+0000","last_change":"2024-04-15T15:23:55.265207+0000","last_active":"2024-04-15T15:23:56.417763+0000","last_peered":"2024-04-15T15:23:56.417763+0000","last_clean":"2024-04-15T15:23:56.417763+0000","last_became_active":"2024-04-15T15:23:55.264857+0000","last_became_peered":"2024-04-15T15:23:55.264857+0000","last_unstale":"2024-04-15T15:23:56.417763+0000","last_undegraded":"2024-04-15T15:23:56.417763+0000","last_fullsized":"2024-04-15T15:23:56.417763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:35.342060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416683+0000","last_change":"2024-04-15T15:23:55.270590+0000","last_active":"2024-04-15T15:23:56.416683+0000","last_peered":"2024-04-15T15:23:56.416683+0000","last_clean":"2024-04-15T15:23:56.416683+0000","last_became_active":"2024-04-15T15:23:55.267976+0000","last_became_peered":"2024-04-15T15:23:55.267976+0000","last_unstale":"2024-04-15T15:23:56.416683+0000","last_undegraded":"2024-04-15T15:23:56.416683+0000","last_fullsized":"2024-04-15T15:23:56.416683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:11.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418517+0000","last_change":"2024-04-15T15:23:55.274056+0000","last_active":"2024-04-15T15:23:56.418517+0000","last_peered":"2024-04-15T15:23:56.418517+0000","last_clean":"2024-04-15T15:23:56.418517+0000","last_became_active":"2024-04-15T15:23:55.273746+0000","last_became_peered":"2024-04-15T15:23:55.273746+0000","last_unstale":"2024-04-15T15:23:56.418517+0000","last_undegraded":"2024-04-15T15:23:56.418517+0000","last_fullsized":"2024-04-15T15:23:56.418517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:31:11.922705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262425+0000","last_change":"2024-04-15T15:23:55.272281+0000","last_active":"2024-04-15T15:23:56.262425+0000","last_peered":"2024-04-15T15:23:56.262425+0000","last_clean":"2024-04-15T15:23:56.262425+0000","last_became_active":"2024-04-15T15:23:55.269795+0000","last_became_peered":"2024-04-15T15:23:55.269795+0000","last_unstale":"2024-04-15T15:23:56.262425+0000","last_undegraded":"2024-04-15T15:23:56.262425+0000","last_fullsized":"2024-04-15T15:23:56.262425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263663+0000","last_change":"2024-04-15T15:23:55.269199+0000","last_active":"2024-04-15T15:23:56.263663+0000","last_peered":"2024-04-15T15:23:56.263663+0000","last_clean":"2024-04-15T15:23:56.263663+0000","last_became_active":"2024-04-15T15:23:55.268870+0000","last_became_peered":"2024-04-15T15:23:55.268870+0000","last_unstale":"2024-04-15T15:23:56.263663+0000","last_undegraded":"2024-04-15T15:23:56.263663+0000","last_fullsized":"2024-04-15T15:23:56.263663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418307+0000","last_change":"2024-04-15T15:23:55.263524+0000","last_active":"2024-04-15T15:23:56.418307+0000","last_peered":"2024-04-15T15:23:56.418307+0000","last_clean":"2024-04-15T15:23:56.418307+0000","last_became_active":"2024-04-15T15:23:55.263210+0000","last_became_peered":"2024-04-15T15:23:55.263210+0000","last_unstale":"2024-04-15T15:23:56.418307+0000","last_undegraded":"2024-04-15T15:23:56.418307+0000","last_fullsized":"2024-04-15T15:23:56.418307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:32.154491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415704+0000","last_change":"2024-04-15T15:23:55.272996+0000","last_active":"2024-04-15T15:23:56.415704+0000","last_peered":"2024-04-15T15:23:56.415704+0000","last_clean":"2024-04-15T15:23:56.415704+0000","last_became_active":"2024-04-15T15:23:55.272701+0000","last_became_peered":"2024-04-15T15:23:55.272701+0000","last_unstale":"2024-04-15T15:23:56.415704+0000","last_undegraded":"2024-04-15T15:23:56.415704+0000","last_fullsized":"2024-04-15T15:23:56.415704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417462+0000","last_change":"2024-04-15T15:23:55.260534+0000","last_active":"2024-04-15T15:23:56.417462+0000","last_peered":"2024-04-15T15:23:56.417462+0000","last_clean":"2024-04-15T15:23:56.417462+0000","last_became_active":"2024-04-15T15:23:55.258506+0000","last_became_peered":"2024-04-15T15:23:55.258506+0000","last_unstale":"2024-04-15T15:23:56.417462+0000","last_undegraded":"2024-04-15T15:23:56.417462+0000","last_fullsized":"2024-04-15T15:23:56.417462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255395+0000","last_change":"2024-04-15T15:23:55.265936+0000","last_active":"2024-04-15T15:23:56.255395+0000","last_peered":"2024-04-15T15:23:56.255395+0000","last_clean":"2024-04-15T15:23:56.255395+0000","last_became_active":"2024-04-15T15:23:55.265118+0000","last_became_peered":"2024-04-15T15:23:55.265118+0000","last_unstale":"2024-04-15T15:23:56.255395+0000","last_undegraded":"2024-04-15T15:23:56.255395+0000","last_fullsized":"2024-04-15T15:23:56.255395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.260985+0000","last_change":"2024-04-15T15:23:55.252289+0000","last_active":"2024-04-15T15:23:56.260985+0000","last_peered":"2024-04-15T15:23:56.260985+0000","last_clean":"2024-04-15T15:23:56.260985+0000","last_became_active":"2024-04-15T15:23:55.252007+0000","last_became_peered":"2024-04-15T15:23:55.252007+0000","last_unstale":"2024-04-15T15:23:56.260985+0000","last_undegraded":"2024-04-15T15:23:56.260985+0000","last_fullsized":"2024-04-15T15:23:56.260985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262956+0000","last_change":"2024-04-15T15:23:55.256170+0000","last_active":"2024-04-15T15:23:56.262956+0000","last_peered":"2024-04-15T15:23:56.262956+0000","last_clean":"2024-04-15T15:23:56.262956+0000","last_became_active":"2024-04-15T15:23:55.255896+0000","last_became_peered":"2024-04-15T15:23:55.255896+0000","last_unstale":"2024-04-15T15:23:56.262956+0000","last_undegraded":"2024-04-15T15:23:56.262956+0000","last_fullsized":"2024-04-15T15:23:56.262956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:16:50.418929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417610+0000","last_change":"2024-04-15T15:23:55.269766+0000","last_active":"2024-04-15T15:23:56.417610+0000","last_peered":"2024-04-15T15:23:56.417610+0000","last_clean":"2024-04-15T15:23:56.417610+0000","last_became_active":"2024-04-15T15:23:55.269443+0000","last_became_peered":"2024-04-15T15:23:55.269443+0000","last_unstale":"2024-04-15T15:23:56.417610+0000","last_undegraded":"2024-04-15T15:23:56.417610+0000","last_fullsized":"2024-04-15T15:23:56.417610+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:43:55.932310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419019+0000","last_change":"2024-04-15T15:23:55.265016+0000","last_active":"2024-04-15T15:23:56.419019+0000","last_peered":"2024-04-15T15:23:56.419019+0000","last_clean":"2024-04-15T15:23:56.419019+0000","last_became_active":"2024-04-15T15:23:55.258660+0000","last_became_peered":"2024-04-15T15:23:55.258660+0000","last_unstale":"2024-04-15T15:23:56.419019+0000","last_undegraded":"2024-04-15T15:23:56.419019+0000","last_fullsized":"2024-04-15T15:23:56.419019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:08:08.360698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262025+0000","last_change":"2024-04-15T15:23:55.275816+0000","last_active":"2024-04-15T15:23:56.262025+0000","last_peered":"2024-04-15T15:23:56.262025+0000","last_clean":"2024-04-15T15:23:56.262025+0000","last_became_active":"2024-04-15T15:23:55.274960+0000","last_became_peered":"2024-04-15T15:23:55.274960+0000","last_unstale":"2024-04-15T15:23:56.262025+0000","last_undegraded":"2024-04-15T15:23:56.262025+0000","last_fullsized":"2024-04-15T15:23:56.262025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417084+0000","last_change":"2024-04-15T15:23:55.267469+0000","last_active":"2024-04-15T15:23:56.417084+0000","last_peered":"2024-04-15T15:23:56.417084+0000","last_clean":"2024-04-15T15:23:56.417084+0000","last_became_active":"2024-04-15T15:23:55.266878+0000","last_became_peered":"2024-04-15T15:23:55.266878+0000","last_unstale":"2024-04-15T15:23:56.417084+0000","last_undegraded":"2024-04-15T15:23:56.417084+0000","last_fullsized":"2024-04-15T15:23:56.417084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262363+0000","last_change":"2024-04-15T15:23:55.262231+0000","last_active":"2024-04-15T15:23:56.262363+0000","last_peered":"2024-04-15T15:23:56.262363+0000","last_clean":"2024-04-15T15:23:56.262363+0000","last_became_active":"2024-04-15T15:23:55.261861+0000","last_became_peered":"2024-04-15T15:23:55.261861+0000","last_unstale":"2024-04-15T15:23:56.262363+0000","last_undegraded":"2024-04-15T15:23:56.262363+0000","last_fullsized":"2024-04-15T15:23:56.262363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:38.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417901+0000","last_change":"2024-04-15T15:23:55.275456+0000","last_active":"2024-04-15T15:23:56.417901+0000","last_peered":"2024-04-15T15:23:56.417901+0000","last_clean":"2024-04-15T15:23:56.417901+0000","last_became_active":"2024-04-15T15:23:55.275108+0000","last_became_peered":"2024-04-15T15:23:55.275108+0000","last_unstale":"2024-04-15T15:23:56.417901+0000","last_undegraded":"2024-04-15T15:23:56.417901+0000","last_fullsized":"2024-04-15T15:23:56.417901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:38.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261274+0000","last_change":"2024-04-15T15:23:55.270445+0000","last_active":"2024-04-15T15:23:56.261274+0000","last_peered":"2024-04-15T15:23:56.261274+0000","last_clean":"2024-04-15T15:23:56.261274+0000","last_became_active":"2024-04-15T15:23:55.270131+0000","last_became_peered":"2024-04-15T15:23:55.270131+0000","last_unstale":"2024-04-15T15:23:56.261274+0000","last_undegraded":"2024-04-15T15:23:56.261274+0000","last_fullsized":"2024-04-15T15:23:56.261274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415461+0000","last_change":"2024-04-15T15:23:55.260659+0000","last_active":"2024-04-15T15:23:56.415461+0000","last_peered":"2024-04-15T15:23:56.415461+0000","last_clean":"2024-04-15T15:23:56.415461+0000","last_became_active":"2024-04-15T15:23:55.258739+0000","last_became_peered":"2024-04-15T15:23:55.258739+0000","last_unstale":"2024-04-15T15:23:56.415461+0000","last_undegraded":"2024-04-15T15:23:56.415461+0000","last_fullsized":"2024-04-15T15:23:56.415461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262634+0000","last_change":"2024-04-15T15:23:55.272818+0000","last_active":"2024-04-15T15:23:56.262634+0000","last_peered":"2024-04-15T15:23:56.262634+0000","last_clean":"2024-04-15T15:23:56.262634+0000","last_became_active":"2024-04-15T15:23:55.269858+0000","last_became_peered":"2024-04-15T15:23:55.269858+0000","last_unstale":"2024-04-15T15:23:56.262634+0000","last_undegraded":"2024-04-15T15:23:56.262634+0000","last_fullsized":"2024-04-15T15:23:56.262634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255726+0000","last_change":"2024-04-15T15:23:55.261128+0000","last_active":"2024-04-15T15:23:56.255726+0000","last_peered":"2024-04-15T15:23:56.255726+0000","last_clean":"2024-04-15T15:23:56.255726+0000","last_became_active":"2024-04-15T15:23:55.259248+0000","last_became_peered":"2024-04-15T15:23:55.259248+0000","last_unstale":"2024-04-15T15:23:56.255726+0000","last_undegraded":"2024-04-15T15:23:56.255726+0000","last_fullsized":"2024-04-15T15:23:56.255726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415563+0000","last_change":"2024-04-15T15:23:55.264402+0000","last_active":"2024-04-15T15:23:56.415563+0000","last_peered":"2024-04-15T15:23:56.415563+0000","last_clean":"2024-04-15T15:23:56.415563+0000","last_became_active":"2024-04-15T15:23:55.264083+0000","last_became_peered":"2024-04-15T15:23:55.264083+0000","last_unstale":"2024-04-15T15:23:56.415563+0000","last_undegraded":"2024-04-15T15:23:56.415563+0000","last_fullsized":"2024-04-15T15:23:56.415563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417369+0000","last_change":"2024-04-15T15:23:55.262147+0000","last_active":"2024-04-15T15:23:56.417369+0000","last_peered":"2024-04-15T15:23:56.417369+0000","last_clean":"2024-04-15T15:23:56.417369+0000","last_became_active":"2024-04-15T15:23:55.261524+0000","last_became_peered":"2024-04-15T15:23:55.261524+0000","last_unstale":"2024-04-15T15:23:56.417369+0000","last_undegraded":"2024-04-15T15:23:56.417369+0000","last_fullsized":"2024-04-15T15:23:56.417369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262811+0000","last_change":"2024-04-15T15:23:55.267385+0000","last_active":"2024-04-15T15:23:56.262811+0000","last_peered":"2024-04-15T15:23:56.262811+0000","last_clean":"2024-04-15T15:23:56.262811+0000","last_became_active":"2024-04-15T15:23:55.267045+0000","last_became_peered":"2024-04-15T15:23:55.267045+0000","last_unstale":"2024-04-15T15:23:56.262811+0000","last_undegraded":"2024-04-15T15:23:56.262811+0000","last_fullsized":"2024-04-15T15:23:56.262811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:35.342060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417143+0000","last_change":"2024-04-15T15:23:55.261039+0000","last_active":"2024-04-15T15:23:56.417143+0000","last_peered":"2024-04-15T15:23:56.417143+0000","last_clean":"2024-04-15T15:23:56.417143+0000","last_became_active":"2024-04-15T15:23:55.260620+0000","last_became_peered":"2024-04-15T15:23:55.260620+0000","last_unstale":"2024-04-15T15:23:56.417143+0000","last_undegraded":"2024-04-15T15:23:56.417143+0000","last_fullsized":"2024-04-15T15:23:56.417143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:27:44.245014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261366+0000","last_change":"2024-04-15T15:23:55.271261+0000","last_active":"2024-04-15T15:23:56.261366+0000","last_peered":"2024-04-15T15:23:56.261366+0000","last_clean":"2024-04-15T15:23:56.261366+0000","last_became_active":"2024-04-15T15:23:55.270954+0000","last_became_peered":"2024-04-15T15:23:55.270954+0000","last_unstale":"2024-04-15T15:23:56.261366+0000","last_undegraded":"2024-04-15T15:23:56.261366+0000","last_fullsized":"2024-04-15T15:23:56.261366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417469+0000","last_change":"2024-04-15T15:23:55.263715+0000","last_active":"2024-04-15T15:23:56.417469+0000","last_peered":"2024-04-15T15:23:56.417469+0000","last_clean":"2024-04-15T15:23:56.417469+0000","last_became_active":"2024-04-15T15:23:55.256992+0000","last_became_peered":"2024-04-15T15:23:55.256992+0000","last_unstale":"2024-04-15T15:23:56.417469+0000","last_undegraded":"2024-04-15T15:23:56.417469+0000","last_fullsized":"2024-04-15T15:23:56.417469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:25.936170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416848+0000","last_change":"2024-04-15T15:23:54.250784+0000","last_active":"2024-04-15T15:23:56.416848+0000","last_peered":"2024-04-15T15:23:56.416848+0000","last_clean":"2024-04-15T15:23:56.416848+0000","last_became_active":"2024-04-15T15:23:54.250347+0000","last_became_peered":"2024-04-15T15:23:54.250347+0000","last_unstale":"2024-04-15T15:23:56.416848+0000","last_undegraded":"2024-04-15T15:23:56.416848+0000","last_fullsized":"2024-04-15T15:23:56.416848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417629+0000","last_change":"2024-04-15T15:23:55.262408+0000","last_active":"2024-04-15T15:23:56.417629+0000","last_peered":"2024-04-15T15:23:56.417629+0000","last_clean":"2024-04-15T15:23:56.417629+0000","last_became_active":"2024-04-15T15:23:55.261815+0000","last_became_peered":"2024-04-15T15:23:55.261815+0000","last_unstale":"2024-04-15T15:23:56.417629+0000","last_undegraded":"2024-04-15T15:23:56.417629+0000","last_fullsized":"2024-04-15T15:23:56.417629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261683+0000","last_change":"2024-04-15T15:23:54.239676+0000","last_active":"2024-04-15T15:23:56.261683+0000","last_peered":"2024-04-15T15:23:56.261683+0000","last_clean":"2024-04-15T15:23:56.261683+0000","last_became_active":"2024-04-15T15:23:54.239084+0000","last_became_peered":"2024-04-15T15:23:54.239084+0000","last_unstale":"2024-04-15T15:23:56.261683+0000","last_undegraded":"2024-04-15T15:23:56.261683+0000","last_fullsized":"2024-04-15T15:23:56.261683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415034+0000","last_change":"2024-04-15T15:23:54.241378+0000","last_active":"2024-04-15T15:23:56.415034+0000","last_peered":"2024-04-15T15:23:56.415034+0000","last_clean":"2024-04-15T15:23:56.415034+0000","last_became_active":"2024-04-15T15:23:54.241084+0000","last_became_peered":"2024-04-15T15:23:54.241084+0000","last_unstale":"2024-04-15T15:23:56.415034+0000","last_undegraded":"2024-04-15T15:23:56.415034+0000","last_fullsized":"2024-04-15T15:23:56.415034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:38:14.692145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.414885+0000","last_change":"2024-04-15T15:23:55.257791+0000","last_active":"2024-04-15T15:23:56.414885+0000","last_peered":"2024-04-15T15:23:56.414885+0000","last_clean":"2024-04-15T15:23:56.414885+0000","last_became_active":"2024-04-15T15:23:55.257531+0000","last_became_peered":"2024-04-15T15:23:55.257531+0000","last_unstale":"2024-04-15T15:23:56.414885+0000","last_undegraded":"2024-04-15T15:23:56.414885+0000","last_fullsized":"2024-04-15T15:23:56.414885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261118+0000","last_change":"2024-04-15T15:23:54.235199+0000","last_active":"2024-04-15T15:23:56.261118+0000","last_peered":"2024-04-15T15:23:56.261118+0000","last_clean":"2024-04-15T15:23:56.261118+0000","last_became_active":"2024-04-15T15:23:54.233620+0000","last_became_peered":"2024-04-15T15:23:54.233620+0000","last_unstale":"2024-04-15T15:23:56.261118+0000","last_undegraded":"2024-04-15T15:23:56.261118+0000","last_fullsized":"2024-04-15T15:23:56.261118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255313+0000","last_change":"2024-04-15T15:23:55.261632+0000","last_active":"2024-04-15T15:23:56.255313+0000","last_peered":"2024-04-15T15:23:56.255313+0000","last_clean":"2024-04-15T15:23:56.255313+0000","last_became_active":"2024-04-15T15:23:55.261351+0000","last_became_peered":"2024-04-15T15:23:55.261351+0000","last_unstale":"2024-04-15T15:23:56.255313+0000","last_undegraded":"2024-04-15T15:23:56.255313+0000","last_fullsized":"2024-04-15T15:23:56.255313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263032+0000","last_change":"2024-04-15T15:23:54.237429+0000","last_active":"2024-04-15T15:23:56.263032+0000","last_peered":"2024-04-15T15:23:56.263032+0000","last_clean":"2024-04-15T15:23:56.263032+0000","last_became_active":"2024-04-15T15:23:54.237045+0000","last_became_peered":"2024-04-15T15:23:54.237045+0000","last_unstale":"2024-04-15T15:23:56.263032+0000","last_undegraded":"2024-04-15T15:23:56.263032+0000","last_fullsized":"2024-04-15T15:23:56.263032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262636+0000","last_change":"2024-04-15T15:23:55.263292+0000","last_active":"2024-04-15T15:23:56.262636+0000","last_peered":"2024-04-15T15:23:56.262636+0000","last_clean":"2024-04-15T15:23:56.262636+0000","last_became_active":"2024-04-15T15:23:55.261899+0000","last_became_peered":"2024-04-15T15:23:55.261899+0000","last_unstale":"2024-04-15T15:23:56.262636+0000","last_undegraded":"2024-04-15T15:23:56.262636+0000","last_fullsized":"2024-04-15T15:23:56.262636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:40.716704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262724+0000","last_change":"2024-04-15T15:23:54.242528+0000","last_active":"2024-04-15T15:23:56.262724+0000","last_peered":"2024-04-15T15:23:56.262724+0000","last_clean":"2024-04-15T15:23:56.262724+0000","last_became_active":"2024-04-15T15:23:54.242233+0000","last_became_peered":"2024-04-15T15:23:54.242233+0000","last_unstale":"2024-04-15T15:23:56.262724+0000","last_undegraded":"2024-04-15T15:23:56.262724+0000","last_fullsized":"2024-04-15T15:23:56.262724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417168+0000","last_change":"2024-04-15T15:23:55.268964+0000","last_active":"2024-04-15T15:23:56.417168+0000","last_peered":"2024-04-15T15:23:56.417168+0000","last_clean":"2024-04-15T15:23:56.417168+0000","last_became_active":"2024-04-15T15:23:55.268348+0000","last_became_peered":"2024-04-15T15:23:55.268348+0000","last_unstale":"2024-04-15T15:23:56.417168+0000","last_undegraded":"2024-04-15T15:23:56.417168+0000","last_fullsized":"2024-04-15T15:23:56.417168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255623+0000","last_change":"2024-04-15T15:23:54.238578+0000","last_active":"2024-04-15T15:23:56.255623+0000","last_peered":"2024-04-15T15:23:56.255623+0000","last_clean":"2024-04-15T15:23:56.255623+0000","last_became_active":"2024-04-15T15:23:54.238332+0000","last_became_peered":"2024-04-15T15:23:54.238332+0000","last_unstale":"2024-04-15T15:23:56.255623+0000","last_undegraded":"2024-04-15T15:23:56.255623+0000","last_fullsized":"2024-04-15T15:23:56.255623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418641+0000","last_change":"2024-04-15T15:23:55.266173+0000","last_active":"2024-04-15T15:23:56.418641+0000","last_peered":"2024-04-15T15:23:56.418641+0000","last_clean":"2024-04-15T15:23:56.418641+0000","last_became_active":"2024-04-15T15:23:55.265877+0000","last_became_peered":"2024-04-15T15:23:55.265877+0000","last_unstale":"2024-04-15T15:23:56.418641+0000","last_undegraded":"2024-04-15T15:23:56.418641+0000","last_fullsized":"2024-04-15T15:23:56.418641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:52.386885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262108+0000","last_change":"2024-04-15T15:23:54.240249+0000","last_active":"2024-04-15T15:23:56.262108+0000","last_peered":"2024-04-15T15:23:56.262108+0000","last_clean":"2024-04-15T15:23:56.262108+0000","last_became_active":"2024-04-15T15:23:54.239151+0000","last_became_peered":"2024-04-15T15:23:54.239151+0000","last_unstale":"2024-04-15T15:23:56.262108+0000","last_undegraded":"2024-04-15T15:23:56.262108+0000","last_fullsized":"2024-04-15T15:23:56.262108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417969+0000","last_change":"2024-04-15T15:23:55.266293+0000","last_active":"2024-04-15T15:23:56.417969+0000","last_peered":"2024-04-15T15:23:56.417969+0000","last_clean":"2024-04-15T15:23:56.417969+0000","last_became_active":"2024-04-15T15:23:55.265962+0000","last_became_peered":"2024-04-15T15:23:55.265962+0000","last_unstale":"2024-04-15T15:23:56.417969+0000","last_undegraded":"2024-04-15T15:23:56.417969+0000","last_fullsized":"2024-04-15T15:23:56.417969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:40.334427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417723+0000","last_change":"2024-04-15T15:23:54.233459+0000","last_active":"2024-04-15T15:23:56.417723+0000","last_peered":"2024-04-15T15:23:56.417723+0000","last_clean":"2024-04-15T15:23:56.417723+0000","last_became_active":"2024-04-15T15:23:54.233056+0000","last_became_peered":"2024-04-15T15:23:54.233056+0000","last_unstale":"2024-04-15T15:23:56.417723+0000","last_undegraded":"2024-04-15T15:23:56.417723+0000","last_fullsized":"2024-04-15T15:23:56.417723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:38:14.692145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417725+0000","last_change":"2024-04-15T15:23:55.261166+0000","last_active":"2024-04-15T15:23:56.417725+0000","last_peered":"2024-04-15T15:23:56.417725+0000","last_clean":"2024-04-15T15:23:56.417725+0000","last_became_active":"2024-04-15T15:23:55.260827+0000","last_became_peered":"2024-04-15T15:23:55.260827+0000","last_unstale":"2024-04-15T15:23:56.417725+0000","last_undegraded":"2024-04-15T15:23:56.417725+0000","last_fullsized":"2024-04-15T15:23:56.417725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262944+0000","last_change":"2024-04-15T15:23:54.233681+0000","last_active":"2024-04-15T15:23:56.262944+0000","last_peered":"2024-04-15T15:23:56.262944+0000","last_clean":"2024-04-15T15:23:56.262944+0000","last_became_active":"2024-04-15T15:23:54.233376+0000","last_became_peered":"2024-04-15T15:23:54.233376+0000","last_unstale":"2024-04-15T15:23:56.262944+0000","last_undegraded":"2024-04-15T15:23:56.262944+0000","last_fullsized":"2024-04-15T15:23:56.262944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419714+0000","last_change":"2024-04-15T15:23:55.275827+0000","last_active":"2024-04-15T15:23:56.419714+0000","last_peered":"2024-04-15T15:23:56.419714+0000","last_clean":"2024-04-15T15:23:56.419714+0000","last_became_active":"2024-04-15T15:23:55.275511+0000","last_became_peered":"2024-04-15T15:23:55.275511+0000","last_unstale":"2024-04-15T15:23:56.419714+0000","last_undegraded":"2024-04-15T15:23:56.419714+0000","last_fullsized":"2024-04-15T15:23:56.419714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:17:08.438022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261296+0000","last_change":"2024-04-15T15:23:54.237666+0000","last_active":"2024-04-15T15:23:56.261296+0000","last_peered":"2024-04-15T15:23:56.261296+0000","last_clean":"2024-04-15T15:23:56.261296+0000","last_became_active":"2024-04-15T15:23:54.237243+0000","last_became_peered":"2024-04-15T15:23:54.237243+0000","last_unstale":"2024-04-15T15:23:56.261296+0000","last_undegraded":"2024-04-15T15:23:56.261296+0000","last_fullsized":"2024-04-15T15:23:56.261296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262905+0000","last_change":"2024-04-15T15:23:55.264795+0000","last_active":"2024-04-15T15:23:56.262905+0000","last_peered":"2024-04-15T15:23:56.262905+0000","last_clean":"2024-04-15T15:23:56.262905+0000","last_became_active":"2024-04-15T15:23:55.264509+0000","last_became_peered":"2024-04-15T15:23:55.264509+0000","last_unstale":"2024-04-15T15:23:56.262905+0000","last_undegraded":"2024-04-15T15:23:56.262905+0000","last_fullsized":"2024-04-15T15:23:56.262905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416505+0000","last_change":"2024-04-15T15:23:54.233260+0000","last_active":"2024-04-15T15:23:56.416505+0000","last_peered":"2024-04-15T15:23:56.416505+0000","last_clean":"2024-04-15T15:23:56.416505+0000","last_became_active":"2024-04-15T15:23:54.231670+0000","last_became_peered":"2024-04-15T15:23:54.231670+0000","last_unstale":"2024-04-15T15:23:56.416505+0000","last_undegraded":"2024-04-15T15:23:56.416505+0000","last_fullsized":"2024-04-15T15:23:56.416505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:33:24.512549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418111+0000","last_change":"2024-04-15T15:23:55.278880+0000","last_active":"2024-04-15T15:23:56.418111+0000","last_peered":"2024-04-15T15:23:56.418111+0000","last_clean":"2024-04-15T15:23:56.418111+0000","last_became_active":"2024-04-15T15:23:55.278617+0000","last_became_peered":"2024-04-15T15:23:55.278617+0000","last_unstale":"2024-04-15T15:23:56.418111+0000","last_undegraded":"2024-04-15T15:23:56.418111+0000","last_fullsized":"2024-04-15T15:23:56.418111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:16:50.418929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418140+0000","last_change":"2024-04-15T15:23:54.239459+0000","last_active":"2024-04-15T15:23:56.418140+0000","last_peered":"2024-04-15T15:23:56.418140+0000","last_clean":"2024-04-15T15:23:56.418140+0000","last_became_active":"2024-04-15T15:23:54.239033+0000","last_became_peered":"2024-04-15T15:23:54.239033+0000","last_unstale":"2024-04-15T15:23:56.418140+0000","last_undegraded":"2024-04-15T15:23:56.418140+0000","last_fullsized":"2024-04-15T15:23:56.418140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:16.311793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419071+0000","last_change":"2024-04-15T15:23:55.277191+0000","last_active":"2024-04-15T15:23:56.419071+0000","last_peered":"2024-04-15T15:23:56.419071+0000","last_clean":"2024-04-15T15:23:56.419071+0000","last_became_active":"2024-04-15T15:23:55.276803+0000","last_became_peered":"2024-04-15T15:23:55.276803+0000","last_unstale":"2024-04-15T15:23:56.419071+0000","last_undegraded":"2024-04-15T15:23:56.419071+0000","last_fullsized":"2024-04-15T15:23:56.419071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:53:48.793142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417566+0000","last_change":"2024-04-15T15:23:54.230441+0000","last_active":"2024-04-15T15:23:56.417566+0000","last_peered":"2024-04-15T15:23:56.417566+0000","last_clean":"2024-04-15T15:23:56.417566+0000","last_became_active":"2024-04-15T15:23:54.230051+0000","last_became_peered":"2024-04-15T15:23:54.230051+0000","last_unstale":"2024-04-15T15:23:56.417566+0000","last_undegraded":"2024-04-15T15:23:56.417566+0000","last_fullsized":"2024-04-15T15:23:56.417566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:29.938678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418932+0000","last_change":"2024-04-15T15:23:55.281762+0000","last_active":"2024-04-15T15:23:56.418932+0000","last_peered":"2024-04-15T15:23:56.418932+0000","last_clean":"2024-04-15T15:23:56.418932+0000","last_became_active":"2024-04-15T15:23:55.281494+0000","last_became_peered":"2024-04-15T15:23:55.281494+0000","last_unstale":"2024-04-15T15:23:56.418932+0000","last_undegraded":"2024-04-15T15:23:56.418932+0000","last_fullsized":"2024-04-15T15:23:56.418932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:42:09.480734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255780+0000","last_change":"2024-04-15T15:23:54.234786+0000","last_active":"2024-04-15T15:23:56.255780+0000","last_peered":"2024-04-15T15:23:56.255780+0000","last_clean":"2024-04-15T15:23:56.255780+0000","last_became_active":"2024-04-15T15:23:54.234484+0000","last_became_peered":"2024-04-15T15:23:54.234484+0000","last_unstale":"2024-04-15T15:23:56.255780+0000","last_undegraded":"2024-04-15T15:23:56.255780+0000","last_fullsized":"2024-04-15T15:23:56.255780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:05.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262958+0000","last_change":"2024-04-15T15:23:55.267889+0000","last_active":"2024-04-15T15:23:56.262958+0000","last_peered":"2024-04-15T15:23:56.262958+0000","last_clean":"2024-04-15T15:23:56.262958+0000","last_became_active":"2024-04-15T15:23:55.267209+0000","last_became_peered":"2024-04-15T15:23:55.267209+0000","last_unstale":"2024-04-15T15:23:56.262958+0000","last_undegraded":"2024-04-15T15:23:56.262958+0000","last_fullsized":"2024-04-15T15:23:56.262958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:40.334427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416588+0000","last_change":"2024-04-15T15:23:54.237963+0000","last_active":"2024-04-15T15:23:56.416588+0000","last_peered":"2024-04-15T15:23:56.416588+0000","last_clean":"2024-04-15T15:23:56.416588+0000","last_became_active":"2024-04-15T15:23:54.237726+0000","last_became_peered":"2024-04-15T15:23:54.237726+0000","last_unstale":"2024-04-15T15:23:56.416588+0000","last_undegraded":"2024-04-15T15:23:56.416588+0000","last_fullsized":"2024-04-15T15:23:56.416588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:45:06.635874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418333+0000","last_change":"2024-04-15T15:23:55.272911+0000","last_active":"2024-04-15T15:23:56.418333+0000","last_peered":"2024-04-15T15:23:56.418333+0000","last_clean":"2024-04-15T15:23:56.418333+0000","last_became_active":"2024-04-15T15:23:55.272642+0000","last_became_peered":"2024-04-15T15:23:55.272642+0000","last_unstale":"2024-04-15T15:23:56.418333+0000","last_undegraded":"2024-04-15T15:23:56.418333+0000","last_fullsized":"2024-04-15T15:23:56.418333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:27.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417306+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:23:56.417306+0000","last_peered":"2024-04-15T15:23:56.417306+0000","last_clean":"2024-04-15T15:23:56.417306+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:23:56.417306+0000","last_undegraded":"2024-04-15T15:23:56.417306+0000","last_fullsized":"2024-04-15T15:23:56.417306+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261029+0000","last_change":"2024-04-15T15:23:54.232974+0000","last_active":"2024-04-15T15:23:56.261029+0000","last_peered":"2024-04-15T15:23:56.261029+0000","last_clean":"2024-04-15T15:23:56.261029+0000","last_became_active":"2024-04-15T15:23:54.230808+0000","last_became_peered":"2024-04-15T15:23:54.230808+0000","last_unstale":"2024-04-15T15:23:56.261029+0000","last_undegraded":"2024-04-15T15:23:56.261029+0000","last_fullsized":"2024-04-15T15:23:56.261029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:46:30.058785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418655+0000","last_change":"2024-04-15T15:23:55.269058+0000","last_active":"2024-04-15T15:23:56.418655+0000","last_peered":"2024-04-15T15:23:56.418655+0000","last_clean":"2024-04-15T15:23:56.418655+0000","last_became_active":"2024-04-15T15:23:55.268267+0000","last_became_peered":"2024-04-15T15:23:55.268267+0000","last_unstale":"2024-04-15T15:23:56.418655+0000","last_undegraded":"2024-04-15T15:23:56.418655+0000","last_fullsized":"2024-04-15T15:23:56.418655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:30:18.931762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417047+0000","last_change":"2024-04-15T15:23:54.251259+0000","last_active":"2024-04-15T15:23:56.417047+0000","last_peered":"2024-04-15T15:23:56.417047+0000","last_clean":"2024-04-15T15:23:56.417047+0000","last_became_active":"2024-04-15T15:23:54.250985+0000","last_became_peered":"2024-04-15T15:23:54.250985+0000","last_unstale":"2024-04-15T15:23:56.417047+0000","last_undegraded":"2024-04-15T15:23:56.417047+0000","last_fullsized":"2024-04-15T15:23:56.417047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:44:11.419953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418428+0000","last_change":"2024-04-15T15:23:55.267279+0000","last_active":"2024-04-15T15:23:56.418428+0000","last_peered":"2024-04-15T15:23:56.418428+0000","last_clean":"2024-04-15T15:23:56.418428+0000","last_became_active":"2024-04-15T15:23:55.266971+0000","last_became_peered":"2024-04-15T15:23:55.266971+0000","last_unstale":"2024-04-15T15:23:56.418428+0000","last_undegraded":"2024-04-15T15:23:56.418428+0000","last_fullsized":"2024-04-15T15:23:56.418428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417318+0000","last_change":"2024-04-15T15:23:54.234761+0000","last_active":"2024-04-15T15:23:56.417318+0000","last_peered":"2024-04-15T15:23:56.417318+0000","last_clean":"2024-04-15T15:23:56.417318+0000","last_became_active":"2024-04-15T15:23:54.234467+0000","last_became_peered":"2024-04-15T15:23:54.234467+0000","last_unstale":"2024-04-15T15:23:56.417318+0000","last_undegraded":"2024-04-15T15:23:56.417318+0000","last_fullsized":"2024-04-15T15:23:56.417318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:47:42.194686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255347+0000","last_change":"2024-04-15T15:23:55.244630+0000","last_active":"2024-04-15T15:23:56.255347+0000","last_peered":"2024-04-15T15:23:56.255347+0000","last_clean":"2024-04-15T15:23:56.255347+0000","last_became_active":"2024-04-15T15:23:55.244208+0000","last_became_peered":"2024-04-15T15:23:55.244208+0000","last_unstale":"2024-04-15T15:23:56.255347+0000","last_undegraded":"2024-04-15T15:23:56.255347+0000","last_fullsized":"2024-04-15T15:23:56.255347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263182+0000","last_change":"2024-04-15T15:23:54.225774+0000","last_active":"2024-04-15T15:23:56.263182+0000","last_peered":"2024-04-15T15:23:56.263182+0000","last_clean":"2024-04-15T15:23:56.263182+0000","last_became_active":"2024-04-15T15:23:54.223951+0000","last_became_peered":"2024-04-15T15:23:54.223951+0000","last_unstale":"2024-04-15T15:23:56.263182+0000","last_undegraded":"2024-04-15T15:23:56.263182+0000","last_fullsized":"2024-04-15T15:23:56.263182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417834+0000","last_change":"2024-04-15T15:23:55.261370+0000","last_active":"2024-04-15T15:23:56.417834+0000","last_peered":"2024-04-15T15:23:56.417834+0000","last_clean":"2024-04-15T15:23:56.417834+0000","last_became_active":"2024-04-15T15:23:55.260930+0000","last_became_peered":"2024-04-15T15:23:55.260930+0000","last_unstale":"2024-04-15T15:23:56.417834+0000","last_undegraded":"2024-04-15T15:23:56.417834+0000","last_fullsized":"2024-04-15T15:23:56.417834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:54.270601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255534+0000","last_change":"2024-04-15T15:23:54.223225+0000","last_active":"2024-04-15T15:23:56.255534+0000","last_peered":"2024-04-15T15:23:56.255534+0000","last_clean":"2024-04-15T15:23:56.255534+0000","last_became_active":"2024-04-15T15:23:54.222903+0000","last_became_peered":"2024-04-15T15:23:54.222903+0000","last_unstale":"2024-04-15T15:23:56.255534+0000","last_undegraded":"2024-04-15T15:23:56.255534+0000","last_fullsized":"2024-04-15T15:23:56.255534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:51.381976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.264575+0000","last_change":"2024-04-15T15:23:55.270383+0000","last_active":"2024-04-15T15:23:56.264575+0000","last_peered":"2024-04-15T15:23:56.264575+0000","last_clean":"2024-04-15T15:23:56.264575+0000","last_became_active":"2024-04-15T15:23:55.267157+0000","last_became_peered":"2024-04-15T15:23:55.267157+0000","last_unstale":"2024-04-15T15:23:56.264575+0000","last_undegraded":"2024-04-15T15:23:56.264575+0000","last_fullsized":"2024-04-15T15:23:56.264575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:27.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419141+0000","last_change":"2024-04-15T15:23:54.251497+0000","last_active":"2024-04-15T15:23:56.419141+0000","last_peered":"2024-04-15T15:23:56.419141+0000","last_clean":"2024-04-15T15:23:56.419141+0000","last_became_active":"2024-04-15T15:23:54.250660+0000","last_became_peered":"2024-04-15T15:23:54.250660+0000","last_unstale":"2024-04-15T15:23:56.419141+0000","last_undegraded":"2024-04-15T15:23:56.419141+0000","last_fullsized":"2024-04-15T15:23:56.419141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:14.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255397+0000","last_change":"2024-04-15T15:23:55.249702+0000","last_active":"2024-04-15T15:23:56.255397+0000","last_peered":"2024-04-15T15:23:56.255397+0000","last_clean":"2024-04-15T15:23:56.255397+0000","last_became_active":"2024-04-15T15:23:55.249084+0000","last_became_peered":"2024-04-15T15:23:55.249084+0000","last_unstale":"2024-04-15T15:23:56.255397+0000","last_undegraded":"2024-04-15T15:23:56.255397+0000","last_fullsized":"2024-04-15T15:23:56.255397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255845+0000","last_change":"2024-04-15T15:23:54.241520+0000","last_active":"2024-04-15T15:23:56.255845+0000","last_peered":"2024-04-15T15:23:56.255845+0000","last_clean":"2024-04-15T15:23:56.255845+0000","last_became_active":"2024-04-15T15:23:54.241103+0000","last_became_peered":"2024-04-15T15:23:54.241103+0000","last_unstale":"2024-04-15T15:23:56.255845+0000","last_undegraded":"2024-04-15T15:23:56.255845+0000","last_fullsized":"2024-04-15T15:23:56.255845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:44:11.419953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415819+0000","last_change":"2024-04-15T15:23:55.265567+0000","last_active":"2024-04-15T15:23:56.415819+0000","last_peered":"2024-04-15T15:23:56.415819+0000","last_clean":"2024-04-15T15:23:56.415819+0000","last_became_active":"2024-04-15T15:23:55.262905+0000","last_became_peered":"2024-04-15T15:23:55.262905+0000","last_unstale":"2024-04-15T15:23:56.415819+0000","last_undegraded":"2024-04-15T15:23:56.415819+0000","last_fullsized":"2024-04-15T15:23:56.415819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263461+0000","last_change":"2024-04-15T15:23:54.252083+0000","last_active":"2024-04-15T15:23:56.263461+0000","last_peered":"2024-04-15T15:23:56.263461+0000","last_clean":"2024-04-15T15:23:56.263461+0000","last_became_active":"2024-04-15T15:23:54.251796+0000","last_became_peered":"2024-04-15T15:23:54.251796+0000","last_unstale":"2024-04-15T15:23:56.263461+0000","last_undegraded":"2024-04-15T15:23:56.263461+0000","last_fullsized":"2024-04-15T15:23:56.263461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:46:30.058785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416720+0000","last_change":"2024-04-15T15:23:55.257995+0000","last_active":"2024-04-15T15:23:56.416720+0000","last_peered":"2024-04-15T15:23:56.416720+0000","last_clean":"2024-04-15T15:23:56.416720+0000","last_became_active":"2024-04-15T15:23:55.257718+0000","last_became_peered":"2024-04-15T15:23:55.257718+0000","last_unstale":"2024-04-15T15:23:56.416720+0000","last_undegraded":"2024-04-15T15:23:56.416720+0000","last_fullsized":"2024-04-15T15:23:56.416720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:08:29.731069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416748+0000","last_change":"2024-04-15T15:23:54.250977+0000","last_active":"2024-04-15T15:23:56.416748+0000","last_peered":"2024-04-15T15:23:56.416748+0000","last_clean":"2024-04-15T15:23:56.416748+0000","last_became_active":"2024-04-15T15:23:54.250302+0000","last_became_peered":"2024-04-15T15:23:54.250302+0000","last_unstale":"2024-04-15T15:23:56.416748+0000","last_undegraded":"2024-04-15T15:23:56.416748+0000","last_fullsized":"2024-04-15T15:23:56.416748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:05.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418747+0000","last_change":"2024-04-15T15:23:55.260220+0000","last_active":"2024-04-15T15:23:56.418747+0000","last_peered":"2024-04-15T15:23:56.418747+0000","last_clean":"2024-04-15T15:23:56.418747+0000","last_became_active":"2024-04-15T15:23:55.259361+0000","last_became_peered":"2024-04-15T15:23:55.259361+0000","last_unstale":"2024-04-15T15:23:56.418747+0000","last_undegraded":"2024-04-15T15:23:56.418747+0000","last_fullsized":"2024-04-15T15:23:56.418747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:06.899022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255405+0000","last_change":"2024-04-15T15:23:54.236256+0000","last_active":"2024-04-15T15:23:56.255405+0000","last_peered":"2024-04-15T15:23:56.255405+0000","last_clean":"2024-04-15T15:23:56.255405+0000","last_became_active":"2024-04-15T15:23:54.235919+0000","last_became_peered":"2024-04-15T15:23:54.235919+0000","last_unstale":"2024-04-15T15:23:56.255405+0000","last_undegraded":"2024-04-15T15:23:56.255405+0000","last_fullsized":"2024-04-15T15:23:56.255405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415070+0000","last_change":"2024-04-15T15:23:55.258682+0000","last_active":"2024-04-15T15:23:56.415070+0000","last_peered":"2024-04-15T15:23:56.415070+0000","last_clean":"2024-04-15T15:23:56.415070+0000","last_became_active":"2024-04-15T15:23:55.258183+0000","last_became_peered":"2024-04-15T15:23:55.258183+0000","last_unstale":"2024-04-15T15:23:56.415070+0000","last_undegraded":"2024-04-15T15:23:56.415070+0000","last_fullsized":"2024-04-15T15:23:56.415070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255301+0000","last_change":"2024-04-15T15:23:54.233682+0000","last_active":"2024-04-15T15:23:56.255301+0000","last_peered":"2024-04-15T15:23:56.255301+0000","last_clean":"2024-04-15T15:23:56.255301+0000","last_became_active":"2024-04-15T15:23:54.233373+0000","last_became_peered":"2024-04-15T15:23:54.233373+0000","last_unstale":"2024-04-15T15:23:56.255301+0000","last_undegraded":"2024-04-15T15:23:56.255301+0000","last_fullsized":"2024-04-15T15:23:56.255301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416048+0000","last_change":"2024-04-15T15:23:55.272358+0000","last_active":"2024-04-15T15:23:56.416048+0000","last_peered":"2024-04-15T15:23:56.416048+0000","last_clean":"2024-04-15T15:23:56.416048+0000","last_became_active":"2024-04-15T15:23:55.271660+0000","last_became_peered":"2024-04-15T15:23:55.271660+0000","last_unstale":"2024-04-15T15:23:56.416048+0000","last_undegraded":"2024-04-15T15:23:56.416048+0000","last_fullsized":"2024-04-15T15:23:56.416048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419745+0000","last_change":"2024-04-15T15:23:54.251693+0000","last_active":"2024-04-15T15:23:56.419745+0000","last_peered":"2024-04-15T15:23:56.419745+0000","last_clean":"2024-04-15T15:23:56.419745+0000","last_became_active":"2024-04-15T15:23:54.250255+0000","last_became_peered":"2024-04-15T15:23:54.250255+0000","last_unstale":"2024-04-15T15:23:56.419745+0000","last_undegraded":"2024-04-15T15:23:56.419745+0000","last_fullsized":"2024-04-15T15:23:56.419745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:08:28.726161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261853+0000","last_change":"2024-04-15T15:23:55.278856+0000","last_active":"2024-04-15T15:23:56.261853+0000","last_peered":"2024-04-15T15:23:56.261853+0000","last_clean":"2024-04-15T15:23:56.261853+0000","last_became_active":"2024-04-15T15:23:55.278605+0000","last_became_peered":"2024-04-15T15:23:55.278605+0000","last_unstale":"2024-04-15T15:23:56.261853+0000","last_undegraded":"2024-04-15T15:23:56.261853+0000","last_fullsized":"2024-04-15T15:23:56.261853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.264390+0000","last_change":"2024-04-15T15:23:55.260285+0000","last_active":"2024-04-15T15:23:56.264390+0000","last_peered":"2024-04-15T15:23:56.264390+0000","last_clean":"2024-04-15T15:23:56.264390+0000","last_became_active":"2024-04-15T15:23:55.260016+0000","last_became_peered":"2024-04-15T15:23:55.260016+0000","last_unstale":"2024-04-15T15:23:56.264390+0000","last_undegraded":"2024-04-15T15:23:56.264390+0000","last_fullsized":"2024-04-15T15:23:56.264390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:58.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418779+0000","last_change":"2024-04-15T15:23:54.250621+0000","last_active":"2024-04-15T15:23:56.418779+0000","last_peered":"2024-04-15T15:23:56.418779+0000","last_clean":"2024-04-15T15:23:56.418779+0000","last_became_active":"2024-04-15T15:23:54.250204+0000","last_became_peered":"2024-04-15T15:23:54.250204+0000","last_unstale":"2024-04-15T15:23:56.418779+0000","last_undegraded":"2024-04-15T15:23:56.418779+0000","last_fullsized":"2024-04-15T15:23:56.418779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:54:51.507224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262544+0000","last_change":"2024-04-15T15:23:55.263054+0000","last_active":"2024-04-15T15:23:56.262544+0000","last_peered":"2024-04-15T15:23:56.262544+0000","last_clean":"2024-04-15T15:23:56.262544+0000","last_became_active":"2024-04-15T15:23:55.262742+0000","last_became_peered":"2024-04-15T15:23:55.262742+0000","last_unstale":"2024-04-15T15:23:56.262544+0000","last_undegraded":"2024-04-15T15:23:56.262544+0000","last_fullsized":"2024-04-15T15:23:56.262544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:30:18.931762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418457+0000","last_change":"2024-04-15T15:23:54.250572+0000","last_active":"2024-04-15T15:23:56.418457+0000","last_peered":"2024-04-15T15:23:56.418457+0000","last_clean":"2024-04-15T15:23:56.418457+0000","last_became_active":"2024-04-15T15:23:54.250229+0000","last_became_peered":"2024-04-15T15:23:54.250229+0000","last_unstale":"2024-04-15T15:23:56.418457+0000","last_undegraded":"2024-04-15T15:23:56.418457+0000","last_fullsized":"2024-04-15T15:23:56.418457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:15.052794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416985+0000","last_change":"2024-04-15T15:23:55.269263+0000","last_active":"2024-04-15T15:23:56.416985+0000","last_peered":"2024-04-15T15:23:56.416985+0000","last_clean":"2024-04-15T15:23:56.416985+0000","last_became_active":"2024-04-15T15:23:55.268706+0000","last_became_peered":"2024-04-15T15:23:55.268706+0000","last_unstale":"2024-04-15T15:23:56.416985+0000","last_undegraded":"2024-04-15T15:23:56.416985+0000","last_fullsized":"2024-04-15T15:23:56.416985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.057703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415538+0000","last_change":"2024-04-15T15:23:54.229794+0000","last_active":"2024-04-15T15:23:56.415538+0000","last_peered":"2024-04-15T15:23:56.415538+0000","last_clean":"2024-04-15T15:23:56.415538+0000","last_became_active":"2024-04-15T15:23:54.229509+0000","last_became_peered":"2024-04-15T15:23:54.229509+0000","last_unstale":"2024-04-15T15:23:56.415538+0000","last_undegraded":"2024-04-15T15:23:56.415538+0000","last_fullsized":"2024-04-15T15:23:56.415538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:49:04.203638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415289+0000","last_change":"2024-04-15T15:23:55.274587+0000","last_active":"2024-04-15T15:23:56.415289+0000","last_peered":"2024-04-15T15:23:56.415289+0000","last_clean":"2024-04-15T15:23:56.415289+0000","last_became_active":"2024-04-15T15:23:55.274127+0000","last_became_peered":"2024-04-15T15:23:55.274127+0000","last_unstale":"2024-04-15T15:23:56.415289+0000","last_undegraded":"2024-04-15T15:23:56.415289+0000","last_fullsized":"2024-04-15T15:23:56.415289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417220+0000","last_change":"2024-04-15T15:23:54.236884+0000","last_active":"2024-04-15T15:23:56.417220+0000","last_peered":"2024-04-15T15:23:56.417220+0000","last_clean":"2024-04-15T15:23:56.417220+0000","last_became_active":"2024-04-15T15:23:54.236583+0000","last_became_peered":"2024-04-15T15:23:54.236583+0000","last_unstale":"2024-04-15T15:23:56.417220+0000","last_undegraded":"2024-04-15T15:23:56.417220+0000","last_fullsized":"2024-04-15T15:23:56.417220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:24:00.224 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-15T15:24:00.225 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-15T15:24:00.225 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-15T15:24:00.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-15T15:24:00.225 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-15T15:24:00.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.476+0000 7f805e000640 1 Processor -- start 2024-04-15T15:24:00.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.476+0000 7f805e000640 1 -- start start 2024-04-15T15:24:00.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.477+0000 7f805e000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8058072660 0x7f805806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f805e000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f805815dbb0 con 0x7f805815cbe0 2024-04-15T15:24:00.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f805e000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f805815d8d0 con 0x7f80580cee00 2024-04-15T15:24:00.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f805e000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f805806f4f0 con 0x7f8058072660 2024-04-15T15:24:00.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f805d600640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f805815cbe0 0x7f805815d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33406/0 (socket says 172.21.15.150:33406) 2024-04-15T15:24:00.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f804fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8058072660 0x7f805806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f805d600640 1 -- 172.21.15.150:0/4228600638 learned_addr learned my addr 172.21.15.150:0/4228600638 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:00.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f804fe00640 1 -- 172.21.15.150:0/4228600638 >> v1:172.21.15.187:6789/0 conn(0x7f80580cee00 legacy=0x7f8058071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:00.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f804fe00640 1 -- 172.21.15.150:0/4228600638 >> v1:172.21.15.150:6789/0 conn(0x7f805815cbe0 legacy=0x7f805815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f804fe00640 1 -- 172.21.15.150:0/4228600638 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805806dde0 con 0x7f8058072660 2024-04-15T15:24:00.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.478+0000 7f8057e00640 1 -- 172.21.15.150:0/4228600638 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180175737 0 0) 0x7f805815dbb0 con 0x7f805815cbe0 2024-04-15T15:24:00.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.480+0000 7f804fe00640 1 --2- 172.21.15.150:0/4228600638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8058072660 0x7f805806e8c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f80480097a0 tx=0x7f804802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7777a4b0c5dbda server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:00.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.480+0000 7f8057e00640 1 -- 172.21.15.150:0/4228600638 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048034070 con 0x7f8058072660 2024-04-15T15:24:00.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.480+0000 7f8057e00640 1 -- 172.21.15.150:0/4228600638 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8048030610 con 0x7f8058072660 2024-04-15T15:24:00.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.480+0000 7f8057e00640 1 -- 172.21.15.150:0/4228600638 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480309e0 con 0x7f8058072660 2024-04-15T15:24:00.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.480+0000 7f805e000640 1 -- 172.21.15.150:0/4228600638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8058072660 msgr2=0x7f805806e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.480+0000 7f805e000640 1 --2- 172.21.15.150:0/4228600638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8058072660 0x7f805806e8c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f80480097a0 tx=0x7f804802b960 comp rx=0 tx=0).stop 2024-04-15T15:24:00.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.481+0000 7f805e000640 1 -- 172.21.15.150:0/4228600638 shutdown_connections 2024-04-15T15:24:00.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.481+0000 7f805e000640 1 --2- 172.21.15.150:0/4228600638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8058072660 0x7f805806e8c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.481+0000 7f805e000640 1 -- 172.21.15.150:0/4228600638 >> 172.21.15.150:0/4228600638 conn(0x7f805806b930 msgr2=0x7f805806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:00.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.481+0000 7f805e000640 1 -- 172.21.15.150:0/4228600638 shutdown_connections 2024-04-15T15:24:00.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.481+0000 7f805e000640 1 -- 172.21.15.150:0/4228600638 wait complete. 2024-04-15T15:24:00.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.483+0000 7f805e000640 1 Processor -- start 2024-04-15T15:24:00.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.483+0000 7f805e000640 1 -- start start 2024-04-15T15:24:00.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.484+0000 7f805e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8058072660 0x7f80581437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.484+0000 7f805e000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80580cee00 0x7f8058154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.484+0000 7f805e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805815cbe0 0x7f8058152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.484+0000 7f805e000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f805815dda0 con 0x7f805815cbe0 2024-04-15T15:24:00.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.484+0000 7f805e000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f805806e3a0 con 0x7f80580cee00 2024-04-15T15:24:00.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.484+0000 7f805e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f805815df20 con 0x7f8058072660 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.484+0000 7f805d600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805815cbe0 0x7f8058152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805d600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805815cbe0 0x7f8058152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44408/0 (socket says 172.21.15.150:44408) 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805d600640 1 -- 172.21.15.150:0/999160776 learned_addr learned my addr 172.21.15.150:0/999160776 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f804fe00640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80580cee00 0x7f8058154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805cc00640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8058072660 0x7f80581437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805d600640 1 -- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8058072660 msgr2=0x7f80581437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805d600640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8058072660 0x7f80581437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805d600640 1 -- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80580cee00 msgr2=0x7f8058154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805d600640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80580cee00 0x7f8058154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805d600640 1 -- 172.21.15.150:0/999160776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805815a880 con 0x7f805815cbe0 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f804fe00640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80580cee00 0x7f8058154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:00.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f805d600640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805815cbe0 0x7f8058152ab0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f805000c730 tx=0x7f805000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:00.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.485+0000 7f8055600640 1 -- 172.21.15.150:0/999160776 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050018070 con 0x7f805815cbe0 2024-04-15T15:24:00.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.486+0000 7f8055600640 1 -- 172.21.15.150:0/999160776 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f805000ce40 con 0x7f805815cbe0 2024-04-15T15:24:00.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.486+0000 7f8055600640 1 -- 172.21.15.150:0/999160776 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805000a860 con 0x7f805815cbe0 2024-04-15T15:24:00.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.486+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8058155510 con 0x7f805815cbe0 2024-04-15T15:24:00.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.486+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8058155ad0 con 0x7f805815cbe0 2024-04-15T15:24:00.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.487+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805815aa50 con 0x7f805815cbe0 2024-04-15T15:24:00.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.488+0000 7f8055600640 1 -- 172.21.15.150:0/999160776 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f805000aa00 con 0x7f805815cbe0 2024-04-15T15:24:00.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.489+0000 7f8055600640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f803406c8f0 0x7f803406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.489+0000 7f8055600640 1 -- 172.21.15.150:0/999160776 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f805008b1f0 con 0x7f805815cbe0 2024-04-15T15:24:00.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.489+0000 7f805cc00640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f803406c8f0 0x7f803406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.491+0000 7f805cc00640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f803406c8f0 0x7f803406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f80581579f0 tx=0x7f8040009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:00.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.493+0000 7f8055600640 1 -- 172.21.15.150:0/999160776 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8050059580 con 0x7f805815cbe0 2024-04-15T15:24:00.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.743+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8058156dd0 con 0x7f805815cbe0 2024-04-15T15:24:00.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.752+0000 7f8055600640 1 -- 172.21.15.150:0/999160776 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174140 (secure 0 0 0) 0x7f8050058e80 con 0x7f805815cbe0 2024-04-15T15:24:00.755 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:00.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f803406c8f0 msgr2=0x7f803406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f803406c8f0 0x7f803406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f80581579f0 tx=0x7f8040009040 comp rx=0 tx=0).stop 2024-04-15T15:24:00.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805815cbe0 msgr2=0x7f8058152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805815cbe0 0x7f8058152ab0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f805000c730 tx=0x7f805000cbf0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 shutdown_connections 2024-04-15T15:24:00.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805815cbe0 0x7f8058152ab0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80580cee00 0x7f8058154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f803406c8f0 0x7f803406edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.757+0000 7f805e000640 1 --2- 172.21.15.150:0/999160776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8058072660 0x7f80581437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.758+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 >> 172.21.15.150:0/999160776 conn(0x7f805806b930 msgr2=0x7f80580b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:00.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.758+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 shutdown_connections 2024-04-15T15:24:00.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.758+0000 7f805e000640 1 -- 172.21.15.150:0/999160776 wait complete. 2024-04-15T15:24:00.776 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":105751222},{"type":"v1","addr":"172.21.15.187:6833","nonce":105751222}]},"active_addr":"172.21.15.187:6833/105751222","active_change":"2024-04-15T15:23:35.191589+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":1409750125}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":714530912}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":4034754873}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":1121052038}]}]} 2024-04-15T15:24:00.780 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-15T15:24:00.780 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-15T15:24:00.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:00.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.987+0000 7f692ee00640 1 Processor -- start 2024-04-15T15:24:00.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.988+0000 7f692ee00640 1 -- start start 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.989+0000 7f692ee00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f69280b1140 0x7f692815f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.989+0000 7f692ee00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f69280b6760 0x7f6928071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.989+0000 7f692ee00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69280b6e10 con 0x7f6928070cd0 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.989+0000 7f692ee00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f69280b7990 con 0x7f69280b1140 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.989+0000 7f692ee00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f69280b70f0 con 0x7f69280b6760 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f6927e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f69280b1140 0x7f692815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f6927400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6928070cd0 0x7f69280b6050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33408/0 (socket says 172.21.15.150:33408) 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f6927400640 1 -- 172.21.15.150:0/4035499112 learned_addr learned my addr 172.21.15.150:0/4035499112 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f692ce00640 1 --2- 172.21.15.150:0/4035499112 >> v2:172.21.15.187:3300/0 conn(0x7f69280b6760 0x7f6928071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f6927e00640 1 -- 172.21.15.150:0/4035499112 >> v2:172.21.15.187:3300/0 conn(0x7f69280b6760 msgr2=0x7f6928071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:00.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f6927e00640 1 --2- 172.21.15.150:0/4035499112 >> v2:172.21.15.187:3300/0 conn(0x7f69280b6760 0x7f6928071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f6927e00640 1 -- 172.21.15.150:0/4035499112 >> v1:172.21.15.150:6789/0 conn(0x7f6928070cd0 legacy=0x7f69280b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f6926a00640 1 -- 172.21.15.150:0/4035499112 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2102383525 0 0) 0x7f69280b6e10 con 0x7f6928070cd0 2024-04-15T15:24:00.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.990+0000 7f6927e00640 1 -- 172.21.15.150:0/4035499112 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69280b73d0 con 0x7f69280b1140 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.991+0000 7f6927e00640 1 --2- 172.21.15.150:0/4035499112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b1140 0x7f692815f130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f69280b5500 tx=0x7f692002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab09b59b68c6375 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f6926a00640 1 -- 172.21.15.150:0/4035499112 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920039070 con 0x7f69280b1140 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f6926a00640 1 -- 172.21.15.150:0/4035499112 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692002cbf0 con 0x7f69280b1140 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f6926a00640 1 -- 172.21.15.150:0/4035499112 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920034440 con 0x7f69280b1140 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f692ee00640 1 -- 172.21.15.150:0/4035499112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b1140 msgr2=0x7f692815f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f692ee00640 1 --2- 172.21.15.150:0/4035499112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b1140 0x7f692815f130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f69280b5500 tx=0x7f692002be40 comp rx=0 tx=0).stop 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f692ee00640 1 -- 172.21.15.150:0/4035499112 shutdown_connections 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f692ee00640 1 --2- 172.21.15.150:0/4035499112 >> v2:172.21.15.187:3300/0 conn(0x7f69280b6760 0x7f6928071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f692ee00640 1 --2- 172.21.15.150:0/4035499112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b1140 0x7f692815f130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f692ee00640 1 -- 172.21.15.150:0/4035499112 >> 172.21.15.150:0/4035499112 conn(0x7f692809aa40 msgr2=0x7f692809ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:00.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f692ee00640 1 -- 172.21.15.150:0/4035499112 shutdown_connections 2024-04-15T15:24:00.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.992+0000 7f692ee00640 1 -- 172.21.15.150:0/4035499112 wait complete. 2024-04-15T15:24:00.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.994+0000 7f692ee00640 1 Processor -- start 2024-04-15T15:24:00.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.994+0000 7f692ee00640 1 -- start start 2024-04-15T15:24:00.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.995+0000 7f692ee00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6928070cd0 0x7f69281f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.995+0000 7f692ee00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280b1140 0x7f6928138a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.995+0000 7f692ee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b6760 0x7f69281f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.995+0000 7f692ee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f692815f670 con 0x7f69280b6760 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.995+0000 7f692ee00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69280d1e10 con 0x7f6928070cd0 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.995+0000 7f692ee00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f692815f7f0 con 0x7f69280b1140 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b6760 0x7f69281f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b6760 0x7f69281f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44418/0 (socket says 172.21.15.150:44418) 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 -- 172.21.15.150:0/969144832 learned_addr learned my addr 172.21.15.150:0/969144832 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f6927e00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6928070cd0 0x7f69281f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f6927400640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280b1140 0x7f6928138a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 -- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280b1140 msgr2=0x7f6928138a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280b1140 0x7f6928138a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 -- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6928070cd0 msgr2=0x7f69281f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6928070cd0 0x7f69281f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 -- 172.21.15.150:0/969144832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69280cac70 con 0x7f69280b6760 2024-04-15T15:24:00.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f6927400640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280b1140 0x7f6928138a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:01.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.996+0000 7f692ce00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b6760 0x7f69281f3dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f691800d5d0 tx=0x7f691800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:01.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.997+0000 7f691fe00640 1 -- 172.21.15.150:0/969144832 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180125f0 con 0x7f69280b6760 2024-04-15T15:24:01.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.997+0000 7f691fe00640 1 -- 172.21.15.150:0/969144832 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6918019070 con 0x7f69280b6760 2024-04-15T15:24:01.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.997+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6928139230 con 0x7f69280b6760 2024-04-15T15:24:01.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.997+0000 7f691fe00640 1 -- 172.21.15.150:0/969144832 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180129f0 con 0x7f69280b6760 2024-04-15T15:24:01.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.997+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69281397f0 con 0x7f69280b6760 2024-04-15T15:24:01.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.998+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6928068c80 con 0x7f69280b6760 2024-04-15T15:24:01.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:00.999+0000 7f691fe00640 1 -- 172.21.15.150:0/969144832 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6918012b90 con 0x7f69280b6760 2024-04-15T15:24:01.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.000+0000 7f691fe00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.000+0000 7f691fe00640 1 -- 172.21.15.150:0/969144832 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f691808bf90 con 0x7f69280b6760 2024-04-15T15:24:01.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.000+0000 7f6927e00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.001+0000 7f6927e00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f690406c8f0 0x7f690406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6920036000 tx=0x7f6920030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:01.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.004+0000 7f691fe00640 1 -- 172.21.15.150:0/969144832 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f691805a330 con 0x7f69280b6760 2024-04-15T15:24:01.131 INFO:tasks.ceph.mds.b.smithi187.stderr:2024-04-15T15:24:01.130+0000 7f606b000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:24:01.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.200+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69280d19c0 con 0x7f69280b6760 2024-04-15T15:24:01.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.202+0000 7f691fe00640 1 -- 172.21.15.150:0/969144832 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7f6918059c30 con 0x7f69280b6760 2024-04-15T15:24:01.204 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:01.205 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:01.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f690406c8f0 msgr2=0x7f690406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f690406c8f0 0x7f690406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6920036000 tx=0x7f6920030000 comp rx=0 tx=0).stop 2024-04-15T15:24:01.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b6760 msgr2=0x7f69281f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b6760 0x7f69281f3dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f691800d5d0 tx=0x7f691800da90 comp rx=0 tx=0).stop 2024-04-15T15:24:01.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 shutdown_connections 2024-04-15T15:24:01.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69280b6760 0x7f69281f3dd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280b1140 0x7f6928138a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 --2- 172.21.15.150:0/969144832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6928070cd0 0x7f69281f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.205+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 >> 172.21.15.150:0/969144832 conn(0x7f692809aa40 msgr2=0x7f69280b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:01.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.206+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 shutdown_connections 2024-04-15T15:24:01.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.206+0000 7f692ee00640 1 -- 172.21.15.150:0/969144832 wait complete. 2024-04-15T15:24:01.227 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-15T15:24:01.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:01.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.435+0000 7fa6f5400640 1 Processor -- start 2024-04-15T15:24:01.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.435+0000 7fa6f5400640 1 -- start start 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.436+0000 7fa6f5400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f00cee00 0x7fa6f006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6f5400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6f0071f50 0x7fa6f00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6f5400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f015dbb0 con 0x7fa6f015cbe0 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6f5400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f006f4f0 con 0x7fa6f0071f50 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6f5400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f00728f0 con 0x7fa6f00cee00 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6e7400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6f0071f50 0x7fa6f00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6e7400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6f0071f50 0x7fa6f00723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44434/0 (socket says 172.21.15.150:44434) 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6e7400640 1 -- 172.21.15.150:0/2555945480 learned_addr learned my addr 172.21.15.150:0/2555945480 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6efe00640 1 --2- 172.21.15.150:0/2555945480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f00cee00 0x7fa6f006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6e7400640 1 -- 172.21.15.150:0/2555945480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f00cee00 msgr2=0x7fa6f006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6e7400640 1 --2- 172.21.15.150:0/2555945480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f00cee00 0x7fa6f006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6e7400640 1 -- 172.21.15.150:0/2555945480 >> v1:172.21.15.187:6789/0 conn(0x7fa6f015cbe0 legacy=0x7fa6f015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:01.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.437+0000 7fa6e7400640 1 -- 172.21.15.150:0/2555945480 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f015d8d0 con 0x7fa6f0071f50 2024-04-15T15:24:01.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.438+0000 7fa6e7400640 1 --2- 172.21.15.150:0/2555945480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f0071f50 0x7fa6f00723b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80098a0 tx=0x7fa6d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d028e21d1792d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:01.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6ef400640 1 -- 172.21.15.150:0/2555945480 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8035020 con 0x7fa6f0071f50 2024-04-15T15:24:01.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6ef400640 1 -- 172.21.15.150:0/2555945480 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d8031640 con 0x7fa6f0071f50 2024-04-15T15:24:01.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6ef400640 1 -- 172.21.15.150:0/2555945480 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d802d040 con 0x7fa6f0071f50 2024-04-15T15:24:01.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6f5400640 1 -- 172.21.15.150:0/2555945480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f0071f50 msgr2=0x7fa6f00723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6f5400640 1 --2- 172.21.15.150:0/2555945480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f0071f50 0x7fa6f00723b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80098a0 tx=0x7fa6d802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6f5400640 1 -- 172.21.15.150:0/2555945480 shutdown_connections 2024-04-15T15:24:01.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6f5400640 1 --2- 172.21.15.150:0/2555945480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f0071f50 0x7fa6f00723b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6f5400640 1 --2- 172.21.15.150:0/2555945480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f00cee00 0x7fa6f006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.439+0000 7fa6f5400640 1 -- 172.21.15.150:0/2555945480 >> 172.21.15.150:0/2555945480 conn(0x7fa6f006b930 msgr2=0x7fa6f006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:01.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.440+0000 7fa6f5400640 1 -- 172.21.15.150:0/2555945480 shutdown_connections 2024-04-15T15:24:01.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.440+0000 7fa6f5400640 1 -- 172.21.15.150:0/2555945480 wait complete. 2024-04-15T15:24:01.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.441+0000 7fa6f5400640 1 Processor -- start 2024-04-15T15:24:01.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.441+0000 7fa6f5400640 1 -- start start 2024-04-15T15:24:01.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.442+0000 7fa6f5400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f0071f50 0x7fa6f014e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.442+0000 7fa6f5400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f00cee00 0x7fa6f01598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.442+0000 7fa6f5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f015cbe0 0x7fa6f0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.442+0000 7fa6f5400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f006e3a0 con 0x7fa6f015cbe0 2024-04-15T15:24:01.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.442+0000 7fa6f5400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f006e0c0 con 0x7fa6f00cee00 2024-04-15T15:24:01.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.442+0000 7fa6f5400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f015dda0 con 0x7fa6f0071f50 2024-04-15T15:24:01.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6f4a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f015cbe0 0x7fa6f0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f0071f50 0x7fa6f014e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6e7400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f00cee00 0x7fa6f01598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6f4a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f015cbe0 0x7fa6f0159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44448/0 (socket says 172.21.15.150:44448) 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6f4a00640 1 -- 172.21.15.150:0/3043259670 learned_addr learned my addr 172.21.15.150:0/3043259670 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6f4a00640 1 -- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f0071f50 msgr2=0x7fa6f014e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6f4a00640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f0071f50 0x7fa6f014e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6f4a00640 1 -- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f00cee00 msgr2=0x7fa6f01598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6f4a00640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f00cee00 0x7fa6f01598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.443+0000 7fa6f4a00640 1 -- 172.21.15.150:0/3043259670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f01e6cc0 con 0x7fa6f015cbe0 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.444+0000 7fa6f4a00640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f015cbe0 0x7fa6f0159df0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800b410 tx=0x7fa6e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.444+0000 7fa6ecc00640 1 -- 172.21.15.150:0/3043259670 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8015030 con 0x7fa6f015cbe0 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.444+0000 7fa6ecc00640 1 -- 172.21.15.150:0/3043259670 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e8004020 con 0x7fa6f015cbe0 2024-04-15T15:24:01.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.444+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f0160130 con 0x7fa6f015cbe0 2024-04-15T15:24:01.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.444+0000 7fa6ecc00640 1 -- 172.21.15.150:0/3043259670 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8004410 con 0x7fa6f015cbe0 2024-04-15T15:24:01.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.444+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f01606f0 con 0x7fa6f015cbe0 2024-04-15T15:24:01.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.445+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f01e6e90 con 0x7fa6f015cbe0 2024-04-15T15:24:01.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.446+0000 7fa6ecc00640 1 -- 172.21.15.150:0/3043259670 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6e8018070 con 0x7fa6f015cbe0 2024-04-15T15:24:01.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.447+0000 7fa6ecc00640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa6cc06c920 0x7fa6cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.447+0000 7fa6ecc00640 1 -- 172.21.15.150:0/3043259670 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e808b3e0 con 0x7fa6f015cbe0 2024-04-15T15:24:01.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.447+0000 7fa6efe00640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa6cc06c920 0x7fa6cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.448+0000 7fa6efe00640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa6cc06c920 0x7fa6cc06ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0003b80 tx=0x7fa6e00098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:01.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.451+0000 7fa6ecc00640 1 -- 172.21.15.150:0/3043259670 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6e8059250 con 0x7fa6f015cbe0 2024-04-15T15:24:01.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.652+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6f0156eb0 con 0x7fa6f015cbe0 2024-04-15T15:24:01.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.654+0000 7fa6ecc00640 1 -- 172.21.15.150:0/3043259670 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7fa6e805d550 con 0x7fa6f015cbe0 2024-04-15T15:24:01.656 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:01.656 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:01.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa6cc06c920 msgr2=0x7fa6cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa6cc06c920 0x7fa6cc06ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0003b80 tx=0x7fa6e00098d0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f015cbe0 msgr2=0x7fa6f0159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f015cbe0 0x7fa6f0159df0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800b410 tx=0x7fa6e800b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 shutdown_connections 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f015cbe0 0x7fa6f0159df0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f00cee00 0x7fa6f01598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa6cc06c920 0x7fa6cc06ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 --2- 172.21.15.150:0/3043259670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f0071f50 0x7fa6f014e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 >> 172.21.15.150:0/3043259670 conn(0x7fa6f006b930 msgr2=0x7fa6f01555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 shutdown_connections 2024-04-15T15:24:01.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.657+0000 7fa6f5400640 1 -- 172.21.15.150:0/3043259670 wait complete. 2024-04-15T15:24:01.678 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-15T15:24:01.679 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-15T15:24:01.679 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-15T15:24:01.679 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-15T15:24:01.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-15T15:24:01.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-15T15:24:01.681 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-15T15:24:01.681 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-15T15:24:01.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.977+0000 7fcc7ea00640 1 Processor -- start 2024-04-15T15:24:01.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.977+0000 7fcc7ea00640 1 -- start start 2024-04-15T15:24:01.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.979+0000 7fcc7ea00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcc700e7710 0x7fcc7004ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.979+0000 7fcc7ea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc70053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.979+0000 7fcc7ea00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc70057d90 con 0x7fcc70056ab0 2024-04-15T15:24:01.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.979+0000 7fcc7ea00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fcc70006260 con 0x7fcc700e7710 2024-04-15T15:24:01.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.979+0000 7fcc7ea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc70006540 con 0x7fcc700589c0 2024-04-15T15:24:01.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.980+0000 7fcc7d600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcc700e7710 0x7fcc7004ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.980+0000 7fcc7d600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcc700e7710 0x7fcc7004ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44456/0 (socket says 172.21.15.150:44456) 2024-04-15T15:24:01.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.980+0000 7fcc7d600640 1 -- 172.21.15.150:0/465784979 learned_addr learned my addr 172.21.15.150:0/465784979 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:01.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.981+0000 7fcc7cc00640 1 --2- 172.21.15.150:0/465784979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc70053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.982+0000 7fcc7d600640 1 -- 172.21.15.150:0/465784979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 msgr2=0x7fcc70053530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:01.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.982+0000 7fcc7d600640 1 --2- 172.21.15.150:0/465784979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc70053530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.982+0000 7fcc7d600640 1 -- 172.21.15.150:0/465784979 >> v1:172.21.15.187:6789/0 conn(0x7fcc70056ab0 legacy=0x7fcc70007020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:01.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.982+0000 7fcc7d600640 1 -- 172.21.15.150:0/465784979 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc70006820 con 0x7fcc700e7710 2024-04-15T15:24:01.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.982+0000 7fcc7cc00640 1 --2- 172.21.15.150:0/465784979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc70053530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:24:01.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.984+0000 7fcc7d600640 1 --2- 172.21.15.150:0/465784979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 0x7fcc7004ca30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009870 tx=0x7fcc6c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d930424a7d75f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:01.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.985+0000 7fcc77e00640 1 -- 172.21.15.150:0/465784979 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0317c0 con 0x7fcc700e7710 2024-04-15T15:24:01.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.985+0000 7f618e400640 1 Processor -- start 2024-04-15T15:24:01.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.985+0000 7fcc77e00640 1 -- 172.21.15.150:0/465784979 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c037070 con 0x7fcc700e7710 2024-04-15T15:24:01.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.985+0000 7fcc77e00640 1 -- 172.21.15.150:0/465784979 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c031b80 con 0x7fcc700e7710 2024-04-15T15:24:01.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.985+0000 7f618e400640 1 -- start start 2024-04-15T15:24:01.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.986+0000 7fcc7ea00640 1 -- 172.21.15.150:0/465784979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 msgr2=0x7fcc7004ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.986+0000 7fcc7ea00640 1 --2- 172.21.15.150:0/465784979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 0x7fcc7004ca30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009870 tx=0x7fcc6c02b860 comp rx=0 tx=0).stop 2024-04-15T15:24:01.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7fcc7ea00640 1 -- 172.21.15.150:0/465784979 shutdown_connections 2024-04-15T15:24:01.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7fcc7ea00640 1 --2- 172.21.15.150:0/465784979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc70053530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7fcc7ea00640 1 --2- 172.21.15.150:0/465784979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 0x7fcc7004ca30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7fcc7ea00640 1 -- 172.21.15.150:0/465784979 >> 172.21.15.150:0/465784979 conn(0x7fcc70058ee0 msgr2=0x7fcc700592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:01.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7f618e400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f61880d4fb0 0x7f6188163090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7f618e400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61880d2d20 con 0x7f6188161fc0 2024-04-15T15:24:01.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7f618e400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61880cff70 con 0x7f61880cf9b0 2024-04-15T15:24:01.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7f618e400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f61880d2fc0 con 0x7f61880d4fb0 2024-04-15T15:24:01.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.986+0000 7f3a1a800640 1 Processor -- start 2024-04-15T15:24:01.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.986+0000 7f3a1a800640 1 -- start start 2024-04-15T15:24:01.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.987+0000 7f6187400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6188161fc0 0x7f61880b7af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33412/0 (socket says 172.21.15.150:33412) 2024-04-15T15:24:01.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.988+0000 7f6187400640 1 -- 172.21.15.150:0/3485884879 learned_addr learned my addr 172.21.15.150:0/3485884879 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:01.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.988+0000 7f3a1a800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3a14166490 0x7f3a14164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.988+0000 7f6187e00640 1 --2- 172.21.15.150:0/3485884879 >> v2:172.21.15.150:3300/0 conn(0x7f61880d4fb0 0x7f6188163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.988+0000 7fcc7ea00640 1 -- 172.21.15.150:0/465784979 shutdown_connections 2024-04-15T15:24:01.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.988+0000 7f3a1a800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a140d1a50 0x7f3a141637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.989+0000 7f3a1a800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a140b7210 con 0x7f3a140d3f70 2024-04-15T15:24:01.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.989+0000 7f3a1a800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f3a140b63b0 con 0x7f3a14166490 2024-04-15T15:24:01.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.989+0000 7f3a1a800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3a140b6690 con 0x7f3a140d1a50 2024-04-15T15:24:01.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.989+0000 7f3a12a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3a140d3f70 0x7f3a140d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33426/0 (socket says 172.21.15.150:33426) 2024-04-15T15:24:01.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.989+0000 7f3a12a00640 1 -- 172.21.15.150:0/195095414 learned_addr learned my addr 172.21.15.150:0/195095414 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:01.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.989+0000 7f3a13400640 1 --2- 172.21.15.150:0/195095414 >> v2:172.21.15.150:3300/0 conn(0x7f3a14166490 0x7f3a14164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f3a13400640 1 -- 172.21.15.150:0/195095414 >> v2:172.21.15.187:3300/0 conn(0x7f3a140d1a50 msgr2=0x7f3a141637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:01.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.989+0000 7fcc7ea00640 1 -- 172.21.15.150:0/465784979 wait complete. 2024-04-15T15:24:01.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f3a13e00640 1 --2- 172.21.15.150:0/195095414 >> v2:172.21.15.187:3300/0 conn(0x7f3a140d1a50 0x7f3a141637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f3a13400640 1 --2- 172.21.15.150:0/195095414 >> v2:172.21.15.187:3300/0 conn(0x7f3a140d1a50 0x7f3a141637a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.991+0000 7f3a13400640 1 -- 172.21.15.150:0/195095414 >> v1:172.21.15.150:6789/0 conn(0x7f3a140d3f70 legacy=0x7f3a140d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.991+0000 7f3a13400640 1 -- 172.21.15.150:0/195095414 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a140b6970 con 0x7f3a14166490 2024-04-15T15:24:01.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f6186000640 1 -- 172.21.15.150:0/3485884879 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999699177 0 0) 0x7f61880cff70 con 0x7f61880cf9b0 2024-04-15T15:24:01.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f6186000640 1 -- 172.21.15.150:0/3485884879 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f616c003520 con 0x7f61880cf9b0 2024-04-15T15:24:01.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f6187e00640 1 -- 172.21.15.150:0/3485884879 >> v1:172.21.15.187:6789/0 conn(0x7f61880cf9b0 legacy=0x7f61880d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f6187e00640 1 -- 172.21.15.150:0/3485884879 >> v1:172.21.15.150:6789/0 conn(0x7f6188161fc0 legacy=0x7f61880b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f6187e00640 1 -- 172.21.15.150:0/3485884879 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61880d32a0 con 0x7f61880d4fb0 2024-04-15T15:24:01.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.990+0000 7f6186000640 1 -- 172.21.15.150:0/3485884879 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894443541 0 0) 0x7f61880d2d20 con 0x7f6188161fc0 2024-04-15T15:24:01.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.992+0000 7fcc7ea00640 1 Processor -- start 2024-04-15T15:24:01.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.992+0000 7f3a13400640 1 --2- 172.21.15.150:0/195095414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a14166490 0x7f3a14164610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3a14163470 tx=0x7f3a0402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde1c09968d40f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:01.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.993+0000 7f3a12000640 1 -- 172.21.15.150:0/195095414 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04035020 con 0x7f3a14166490 2024-04-15T15:24:01.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.993+0000 7f3a12000640 1 -- 172.21.15.150:0/195095414 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a04031640 con 0x7f3a14166490 2024-04-15T15:24:01.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.993+0000 7f3a12000640 1 -- 172.21.15.150:0/195095414 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0402d040 con 0x7f3a14166490 2024-04-15T15:24:01.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.992+0000 7f5f15e00640 1 Processor -- start 2024-04-15T15:24:01.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.992+0000 7f5f15e00640 1 -- start start 2024-04-15T15:24:01.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.993+0000 7f6187e00640 1 --2- 172.21.15.150:0/3485884879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 0x7f6188163090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61740097a0 tx=0x7f617402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50703f54b1140e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:01.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.993+0000 7f9c0ea00640 1 Processor -- start 2024-04-15T15:24:01.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.992+0000 7fcc7ea00640 1 -- start start 2024-04-15T15:24:01.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.993+0000 7fcc7ea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc70056ab0 0x7fcc700e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.993+0000 7fcc7ea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc700de530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7f0bbca00640 1 Processor -- start 2024-04-15T15:24:01.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 0x7fcc700dd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:01.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7ea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcc700579a0 con 0x7fcc700e7710 2024-04-15T15:24:01.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7ea00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7004cf70 con 0x7fcc70056ab0 2024-04-15T15:24:01.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7ea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc70057b20 con 0x7fcc700589c0 2024-04-15T15:24:01.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7f6186000640 1 -- 172.21.15.150:0/3485884879 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174035020 con 0x7f61880d4fb0 2024-04-15T15:24:01.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7f6186000640 1 -- 172.21.15.150:0/3485884879 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6174031830 con 0x7f61880d4fb0 2024-04-15T15:24:01.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7f6186000640 1 -- 172.21.15.150:0/3485884879 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174031c00 con 0x7f61880d4fb0 2024-04-15T15:24:01.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 0x7fcc700dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 0x7fcc700dd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44486/0 (socket says 172.21.15.150:44486) 2024-04-15T15:24:01.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7e000640 1 -- 172.21.15.150:0/709644321 learned_addr learned my addr 172.21.15.150:0/709644321 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:01.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7cc00640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc700de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:01.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7fcc7e000640 1 -- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 msgr2=0x7fcc700de530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:01.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7fcc7e000640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc700de530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:01.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7fcc7e000640 1 -- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc70056ab0 msgr2=0x7fcc700e8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f3a1a800640 1 -- 172.21.15.150:0/195095414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a14166490 msgr2=0x7f3a14164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f3a1a800640 1 --2- 172.21.15.150:0/195095414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a14166490 0x7f3a14164610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3a14163470 tx=0x7f3a0402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.994+0000 7f9c0ea00640 1 -- start start 2024-04-15T15:24:02.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7fcc7e000640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc70056ab0 0x7fcc700e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7fcc7e000640 1 -- 172.21.15.150:0/709644321 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc700e94d0 con 0x7fcc700e7710 2024-04-15T15:24:02.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f5f15e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f100b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f5f15e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5f100b5a20 0x7f5f100b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f5f15e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f100cf270 con 0x7f5f10072950 2024-04-15T15:24:02.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f5f15e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5f10158900 con 0x7f5f100b5a20 2024-04-15T15:24:02.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f5f15e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f100d0350 con 0x7f5f100b3e00 2024-04-15T15:24:02.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7fcc7cc00640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc700589c0 0x7fcc700de530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:02.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f3a1a800640 1 -- 172.21.15.150:0/195095414 shutdown_connections 2024-04-15T15:24:02.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f3a1a800640 1 --2- 172.21.15.150:0/195095414 >> v2:172.21.15.187:3300/0 conn(0x7f3a140d1a50 0x7f3a141637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f3a1a800640 1 --2- 172.21.15.150:0/195095414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a14166490 0x7f3a14164610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f3a1a800640 1 -- 172.21.15.150:0/195095414 >> 172.21.15.150:0/195095414 conn(0x7f3a14073690 msgr2=0x7f3a1406cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f3a1a800640 1 -- 172.21.15.150:0/195095414 shutdown_connections 2024-04-15T15:24:02.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f3a1a800640 1 -- 172.21.15.150:0/195095414 wait complete. 2024-04-15T15:24:02.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f0bbca00640 1 -- start start 2024-04-15T15:24:02.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7fcc7e000640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 0x7fcc700dd730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc78075df0 tx=0x7fcc78069540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f9c0ea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 0x7f9c080d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f9c0ea00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c080d32c0 con 0x7f9c080d2c30 2024-04-15T15:24:02.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f9c0ea00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c080d35a0 con 0x7f9c080d1300 2024-04-15T15:24:02.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.995+0000 7f9c0ea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c080d3090 con 0x7f9c0815ec40 2024-04-15T15:24:02.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f5f15400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5f100b5a20 0x7f5f100b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f5f0fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f100b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f5f15400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5f100b5a20 0x7f5f100b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53754/0 (socket says 172.21.15.150:53754) 2024-04-15T15:24:02.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f5f15400640 1 -- 172.21.15.150:0/2735797080 learned_addr learned my addr 172.21.15.150:0/2735797080 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f9c07400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9c080d2c30 0x7f9c080ba390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33452/0 (socket says 172.21.15.150:33452) 2024-04-15T15:24:02.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f9c07400640 1 -- 172.21.15.150:0/2266060387 learned_addr learned my addr 172.21.15.150:0/2266060387 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:02.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f9c07e00640 1 --2- 172.21.15.150:0/2266060387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 0x7f9c080d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f618e400640 1 -- 172.21.15.150:0/3485884879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 msgr2=0x7f6188163090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7f618e400640 1 --2- 172.21.15.150:0/3485884879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 0x7f6188163090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61740097a0 tx=0x7f617402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7f618e400640 1 -- 172.21.15.150:0/3485884879 shutdown_connections 2024-04-15T15:24:02.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.996+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc780754e0 con 0x7fcc700e7710 2024-04-15T15:24:02.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc780756a0 con 0x7fcc700e7710 2024-04-15T15:24:02.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78093640 con 0x7fcc700e7710 2024-04-15T15:24:02.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7f618e400640 1 --2- 172.21.15.150:0/3485884879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 0x7f6188163090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7f618e400640 1 -- 172.21.15.150:0/3485884879 >> 172.21.15.150:0/3485884879 conn(0x7f6188073480 msgr2=0x7f6188073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7f618e400640 1 -- 172.21.15.150:0/3485884879 shutdown_connections 2024-04-15T15:24:02.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7f618e400640 1 -- 172.21.15.150:0/3485884879 wait complete. 2024-04-15T15:24:02.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7f9c07e00640 1 -- 172.21.15.150:0/2266060387 >> v1:172.21.15.187:6789/0 conn(0x7f9c080d1300 legacy=0x7f9c080b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7f9c07e00640 1 -- 172.21.15.150:0/2266060387 >> v1:172.21.15.150:6789/0 conn(0x7f9c080d2c30 legacy=0x7f9c080ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7f9c07e00640 1 -- 172.21.15.150:0/2266060387 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c080d3880 con 0x7f9c0815ec40 2024-04-15T15:24:02.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc700e81d0 con 0x7fcc700e7710 2024-04-15T15:24:02.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.997+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc700e8790 con 0x7fcc700e7710 2024-04-15T15:24:02.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f867c000640 1 Processor -- start 2024-04-15T15:24:02.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f867c000640 1 -- start start 2024-04-15T15:24:02.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.999+0000 7f3a1a800640 1 Processor -- start 2024-04-15T15:24:02.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.999+0000 7f3a1a800640 1 -- start start 2024-04-15T15:24:02.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.999+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc70005390 con 0x7fcc700e7710 2024-04-15T15:24:02.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f867c000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f867415ec40 0x7f86740d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f867c000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f86740d2c30 0x7f86740ba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f867c000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f86740d1300 0x7f86740d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f867c000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f86740d3090 con 0x7f86740d2c30 2024-04-15T15:24:02.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f867c000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f86740ba8d0 con 0x7f86740d1300 2024-04-15T15:24:02.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f9c07e00640 1 --2- 172.21.15.150:0/2266060387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 0x7f9c080d4360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80097d0 tx=0x7f9bf802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7ffee9fa74d2f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f867c000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86740d32c0 con 0x7f867415ec40 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f5f15400640 1 -- 172.21.15.150:0/2735797080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 msgr2=0x7f5f100b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f5f15400640 1 --2- 172.21.15.150:0/2735797080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f100b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f5f15400640 1 -- 172.21.15.150:0/2735797080 >> v1:172.21.15.150:6789/0 conn(0x7f5f10072950 legacy=0x7f5f100ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f5f15400640 1 -- 172.21.15.150:0/2735797080 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f100cf550 con 0x7f5f100b5a20 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f0bbca00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0bb00e3160 0x7f0bb00e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f0bbca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 0x7f0bb0058050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f0bbca00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb0058590 con 0x7f0bb0053cb0 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f0bbca00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb0007b70 con 0x7f0bb00e3160 2024-04-15T15:24:02.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.998+0000 7f0bbca00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb0058810 con 0x7f0bb003c0a0 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.999+0000 7f0bb6a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 0x7f0bb0058050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.999+0000 7f0bb7e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0bb0053cb0 0x7f0bb0054120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33458/0 (socket says 172.21.15.150:33458) 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.999+0000 7f0bb7e00640 1 -- 172.21.15.150:0/1369533439 learned_addr learned my addr 172.21.15.150:0/1369533439 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f0bb7400640 1 --2- 172.21.15.150:0/1369533439 >> v2:172.21.15.150:3300/0 conn(0x7f0bb00e3160 0x7f0bb00e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f0bb7400640 1 -- 172.21.15.150:0/1369533439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 msgr2=0x7f0bb0058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f0bb7400640 1 --2- 172.21.15.150:0/1369533439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 0x7f0bb0058050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f0bb7400640 1 -- 172.21.15.150:0/1369533439 >> v1:172.21.15.150:6789/0 conn(0x7f0bb0053cb0 legacy=0x7f0bb0054120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f0bb7400640 1 -- 172.21.15.150:0/1369533439 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb0058af0 con 0x7f0bb00e3160 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f0bb6000640 1 -- 172.21.15.150:0/1369533439 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509915176 0 0) 0x7f0bb8050c40 con 0x7f0bb0053cb0 2024-04-15T15:24:02.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f8678c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f86740d2c30 0x7f86740ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:01.999+0000 7f618e400640 1 Processor -- start 2024-04-15T15:24:02.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.000+0000 7f618e400640 1 -- start start 2024-04-15T15:24:02.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f8679600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f867415ec40 0x7f86740d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f618e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61880cf9b0 0x7f61880b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f618e400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 0x7f61880c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f8679600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f867415ec40 0x7f86740d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:41380/0 (socket says 172.21.15.150:41380) 2024-04-15T15:24:02.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f8679600640 1 -- 172.21.15.150:0/1647531976 learned_addr learned my addr 172.21.15.150:0/1647531976 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f618e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6188161fc0 0x7f61881f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f618e400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61880d3aa0 con 0x7f61880d4fb0 2024-04-15T15:24:02.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f618e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61881623a0 con 0x7f6188161fc0 2024-04-15T15:24:02.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f618e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61880d3c20 con 0x7f61880cf9b0 2024-04-15T15:24:02.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f3a1a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a140d1a50 0x7f3a14150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a1a800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a140d3f70 0x7f3a14164610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a1a800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a14166490 0x7f3a14164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.001+0000 7f9c06a00640 1 -- 172.21.15.150:0/2266060387 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8034070 con 0x7f9c0815ec40 2024-04-15T15:24:02.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f9c06a00640 1 -- 172.21.15.150:0/2266060387 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf8030800 con 0x7f9c0815ec40 2024-04-15T15:24:02.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f9c06a00640 1 -- 172.21.15.150:0/2266060387 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8030bd0 con 0x7f9c0815ec40 2024-04-15T15:24:02.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a1a800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3a140d4420 con 0x7f3a140d1a50 2024-04-15T15:24:02.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a1a800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a140b6f30 con 0x7f3a140d3f70 2024-04-15T15:24:02.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a1a800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a140d45a0 con 0x7f3a14166490 2024-04-15T15:24:02.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f867a000640 1 --2- 172.21.15.150:0/1647531976 >> v2:172.21.15.187:3300/0 conn(0x7f86740d1300 0x7f86740d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f6187e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6188161fc0 0x7f61881f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f6187e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6188161fc0 0x7f61881f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53766/0 (socket says 172.21.15.150:53766) 2024-04-15T15:24:02.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f6187e00640 1 -- 172.21.15.150:0/2381920424 learned_addr learned my addr 172.21.15.150:0/2381920424 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a13400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a140d1a50 0x7f3a14150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f6186a00640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 0x7f61880c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f53ec600640 1 Processor -- start 2024-04-15T15:24:02.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a13400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a140d1a50 0x7f3a14150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44512/0 (socket says 172.21.15.150:44512) 2024-04-15T15:24:02.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a12a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a140d3f70 0x7f3a14164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a12a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a140d3f70 0x7f3a14164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53778/0 (socket says 172.21.15.150:53778) 2024-04-15T15:24:02.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f3a12a00640 1 -- 172.21.15.150:0/856068747 learned_addr learned my addr 172.21.15.150:0/856068747 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f3a13e00640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a14166490 0x7f3a14164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2266060387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 msgr2=0x7f9c080d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f9c0ea00640 1 --2- 172.21.15.150:0/2266060387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 0x7f9c080d4360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80097d0 tx=0x7f9bf802b940 comp rx=0 tx=0).stop 2024-04-15T15:24:02.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.002+0000 7f0bb7400640 1 --2- 172.21.15.150:0/1369533439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 0x7f0bb00e7780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0097a0 tx=0x7f0bac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694c9a17230003e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f8678c00640 1 -- 172.21.15.150:0/1647531976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f867415ec40 msgr2=0x7f86740d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2266060387 shutdown_connections 2024-04-15T15:24:02.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f9c0ea00640 1 --2- 172.21.15.150:0/2266060387 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 0x7f9c080d4360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2266060387 >> 172.21.15.150:0/2266060387 conn(0x7f9c08073690 msgr2=0x7f9c0806cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc780937e0 con 0x7fcc700e7710 2024-04-15T15:24:02.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2266060387 shutdown_connections 2024-04-15T15:24:02.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2266060387 wait complete. 2024-04-15T15:24:02.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f8678c00640 1 --2- 172.21.15.150:0/1647531976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f867415ec40 0x7f86740d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f8678c00640 1 -- 172.21.15.150:0/1647531976 >> v2:172.21.15.187:3300/0 conn(0x7f86740d1300 msgr2=0x7f86740d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f8678c00640 1 --2- 172.21.15.150:0/1647531976 >> v2:172.21.15.187:3300/0 conn(0x7f86740d1300 0x7f86740d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f6187e00640 1 -- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61880cf9b0 msgr2=0x7f61880b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f8678c00640 1 -- 172.21.15.150:0/1647531976 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86740d35a0 con 0x7f86740d2c30 2024-04-15T15:24:02.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f8679600640 1 --2- 172.21.15.150:0/1647531976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f867415ec40 0x7f86740d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:02.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f3a13400640 1 -- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a14166490 msgr2=0x7f3a14164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f3a13400640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a14166490 0x7f3a14164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f0bb6000640 1 -- 172.21.15.150:0/1369533439 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac035020 con 0x7f0bb00e3160 2024-04-15T15:24:02.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f0bb6000640 1 -- 172.21.15.150:0/1369533439 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bac031830 con 0x7f0bb00e3160 2024-04-15T15:24:02.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f0bb6000640 1 -- 172.21.15.150:0/1369533439 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac031c00 con 0x7f0bb00e3160 2024-04-15T15:24:02.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f6187e00640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61880cf9b0 0x7f61880b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f3a13400640 1 -- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a140d3f70 msgr2=0x7f3a14164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f3a13400640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a140d3f70 0x7f3a14164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f6187e00640 1 -- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 msgr2=0x7f61880c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f6187e00640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 0x7f61880c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f6187e00640 1 -- 172.21.15.150:0/2381920424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61880b4310 con 0x7f6188161fc0 2024-04-15T15:24:02.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f3a13400640 1 -- 172.21.15.150:0/856068747 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a140b8af0 con 0x7f3a140d1a50 2024-04-15T15:24:02.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f6186a00640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61880d4fb0 0x7f61880c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:24:02.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7fcc75600640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc5806c9c0 0x7fcc5806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f6187e00640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6188161fc0 0x7f61881f7870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f61740097a0 tx=0x7f61740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.004+0000 7f3a13400640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a140d1a50 0x7f3a14150fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3a04033000 tx=0x7f3a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.003+0000 7f53ec600640 1 -- start start 2024-04-15T15:24:02.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f53ec600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4162e90 0x7f53e40b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f53ec600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f53e40afd70 0x7f53e4072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f5f15400640 1 --2- 172.21.15.150:0/2735797080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 0x7f5f100b5e90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009870 tx=0x7f5efc02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72f350a1bcd42a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f0bbca00640 1 -- 172.21.15.150:0/1369533439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 msgr2=0x7f0bb00e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f0bbca00640 1 --2- 172.21.15.150:0/1369533439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 0x7f0bb00e7780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0097a0 tx=0x7f0bac02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f53ec600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40b01d0 con 0x7f53e4072640 2024-04-15T15:24:02.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04040070 con 0x7f3a140d1a50 2024-04-15T15:24:02.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174004230 con 0x7f6188161fc0 2024-04-15T15:24:02.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f9c0ea00640 1 Processor -- start 2024-04-15T15:24:02.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f0bbca00640 1 -- 172.21.15.150:0/1369533439 shutdown_connections 2024-04-15T15:24:02.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f0bbca00640 1 --2- 172.21.15.150:0/1369533439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 0x7f0bb0058050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f0bbca00640 1 --2- 172.21.15.150:0/1369533439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 0x7f0bb00e7780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f0bbca00640 1 -- 172.21.15.150:0/1369533439 >> 172.21.15.150:0/1369533439 conn(0x7f0bb0058ee0 msgr2=0x7f0bb00592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f53ec600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f53e4072a90 con 0x7f53e40afd70 2024-04-15T15:24:02.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f53ec600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f53e40b3900 con 0x7f53e4162e90 2024-04-15T15:24:02.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a04039c70 con 0x7f3a140d1a50 2024-04-15T15:24:02.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04035040 con 0x7f3a140d1a50 2024-04-15T15:24:02.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61740043d0 con 0x7f6188161fc0 2024-04-15T15:24:02.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617403a830 con 0x7f6188161fc0 2024-04-15T15:24:02.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f8678c00640 1 --2- 172.21.15.150:0/1647531976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740d2c30 0x7f86740ba390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f867408dde0 tx=0x7f866002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d4838da6c33998 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f0bbca00640 1 -- 172.21.15.150:0/1369533439 shutdown_connections 2024-04-15T15:24:02.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f0bbca00640 1 -- 172.21.15.150:0/1369533439 wait complete. 2024-04-15T15:24:02.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f5f0f400640 1 -- 172.21.15.150:0/2735797080 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc02f830 con 0x7f5f100b5a20 2024-04-15T15:24:02.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f5f0f400640 1 -- 172.21.15.150:0/2735797080 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc02f9d0 con 0x7f5f100b5a20 2024-04-15T15:24:02.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f5f0f400640 1 -- 172.21.15.150:0/2735797080 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc02fda0 con 0x7f5f100b5a20 2024-04-15T15:24:02.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcc78106e40 con 0x7fcc700e7710 2024-04-15T15:24:02.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7fcc75600640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7fcc58074be0 0x7fcc58077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 --> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc78098e50 con 0x7fcc58074be0 2024-04-15T15:24:02.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcc781094a0 con 0x7fcc700e7710 2024-04-15T15:24:02.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7fcc7d600640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc5806c9c0 0x7fcc5806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a140727d0 con 0x7f3a140d1a50 2024-04-15T15:24:02.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a140b8db0 con 0x7f3a140d1a50 2024-04-15T15:24:02.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7fcc7cc00640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7fcc58074be0 0x7fcc58077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61880b45f0 con 0x7f6188161fc0 2024-04-15T15:24:02.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f53e9c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4162e90 0x7f53e40b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.005+0000 7f9c0ea00640 1 -- start start 2024-04-15T15:24:02.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f9c0ea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c080d1300 0x7f9c080cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f9c0ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c080d2c30 0x7f9c0814db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f9c0ea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 0x7f9c0814e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61880b4e90 con 0x7f6188161fc0 2024-04-15T15:24:02.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f9c0ea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c080d17b0 con 0x7f9c080d2c30 2024-04-15T15:24:02.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f9c0ea00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c080d3e40 con 0x7f9c080d1300 2024-04-15T15:24:02.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f9c0ea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c080d1930 con 0x7f9c0815ec40 2024-04-15T15:24:02.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f53e9c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4162e90 0x7f53e40b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:41402/0 (socket says 172.21.15.150:41402) 2024-04-15T15:24:02.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f53e9c00640 1 -- 172.21.15.150:0/311135257 learned_addr learned my addr 172.21.15.150:0/311135257 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f53e9200640 1 --2- 172.21.15.150:0/311135257 >> v2:172.21.15.150:3300/0 conn(0x7f53e40afd70 0x7f53e4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f53e9c00640 1 -- 172.21.15.150:0/311135257 >> v2:172.21.15.150:3300/0 conn(0x7f53e40afd70 msgr2=0x7f53e4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f53e9c00640 1 --2- 172.21.15.150:0/311135257 >> v2:172.21.15.150:3300/0 conn(0x7f53e40afd70 0x7f53e4072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f53e9c00640 1 -- 172.21.15.150:0/311135257 >> v1:172.21.15.187:6789/0 conn(0x7f53e4072640 legacy=0x7f53e406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f53e9c00640 1 -- 172.21.15.150:0/311135257 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e40b3be0 con 0x7f53e4162e90 2024-04-15T15:24:02.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f9c0ca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 0x7f9c0814e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c080d1300 0x7f9c080cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c080d1300 0x7f9c080cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53786/0 (socket says 172.21.15.150:53786) 2024-04-15T15:24:02.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f9c07e00640 1 -- 172.21.15.150:0/2693078025 learned_addr learned my addr 172.21.15.150:0/2693078025 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6188068c80 con 0x7f6188161fc0 2024-04-15T15:24:02.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f53e9200640 1 --2- 172.21.15.150:0/311135257 >> v2:172.21.15.150:3300/0 conn(0x7f53e40afd70 0x7f53e4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:02.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.006+0000 7f8673e00640 1 -- 172.21.15.150:0/1647531976 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660039070 con 0x7f86740d2c30 2024-04-15T15:24:02.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f8673e00640 1 -- 172.21.15.150:0/1647531976 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866002cbe0 con 0x7f86740d2c30 2024-04-15T15:24:02.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f8673e00640 1 -- 172.21.15.150:0/1647531976 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660034440 con 0x7f86740d2c30 2024-04-15T15:24:02.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.007+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a14068c80 con 0x7f3a140d1a50 2024-04-15T15:24:02.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f9c07e00640 1 -- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 msgr2=0x7f9c0814e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f9c07e00640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0815ec40 0x7f9c0814e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f9c07e00640 1 -- 172.21.15.150:0/2693078025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c080d2c30 msgr2=0x7f9c0814db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f867c000640 1 -- 172.21.15.150:0/1647531976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740d2c30 msgr2=0x7f86740ba390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f867c000640 1 --2- 172.21.15.150:0/1647531976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740d2c30 0x7f86740ba390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f867408dde0 tx=0x7f866002be30 comp rx=0 tx=0).stop 2024-04-15T15:24:02.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7fcc7d600640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc5806c9c0 0x7fcc5806ee80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c036000 tx=0x7fcc6c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f9c07e00640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c080d2c30 0x7f9c0814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f9c07e00640 1 -- 172.21.15.150:0/2693078025 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c080cd490 con 0x7f9c080d1300 2024-04-15T15:24:02.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f5f15e00640 1 -- 172.21.15.150:0/2735797080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 msgr2=0x7f5f100b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f5f15e00640 1 --2- 172.21.15.150:0/2735797080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 0x7f5f100b5e90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009870 tx=0x7f5efc02ba10 comp rx=0 tx=0).stop 2024-04-15T15:24:02.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f5f15e00640 1 -- 172.21.15.150:0/2735797080 shutdown_connections 2024-04-15T15:24:02.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f5f15e00640 1 --2- 172.21.15.150:0/2735797080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 0x7f5f100b5e90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f5f15e00640 1 --2- 172.21.15.150:0/2735797080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f100b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.008+0000 7f5f15e00640 1 -- 172.21.15.150:0/2735797080 >> 172.21.15.150:0/2735797080 conn(0x7f5f10071e30 msgr2=0x7f5f10072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f0bbca00640 1 Processor -- start 2024-04-15T15:24:02.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7fcc7cc00640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7fcc58074be0 0x7fcc58077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f5f15e00640 1 -- 172.21.15.150:0/2735797080 shutdown_connections 2024-04-15T15:24:02.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f0bbca00640 1 -- start start 2024-04-15T15:24:02.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f9c07e00640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c080d1300 0x7f9c080cca80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80098f0 tx=0x7f9bf80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.009+0000 7f53e9c00640 1 --2- 172.21.15.150:0/311135257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4162e90 0x7f53e40b58e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009e50 tx=0x7f53dc02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d20a593c02c0f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f867c000640 1 -- 172.21.15.150:0/1647531976 shutdown_connections 2024-04-15T15:24:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f867c000640 1 --2- 172.21.15.150:0/1647531976 >> v2:172.21.15.187:3300/0 conn(0x7f86740d1300 0x7f86740d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f867c000640 1 --2- 172.21.15.150:0/1647531976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740d2c30 0x7f86740ba390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f867c000640 1 --2- 172.21.15.150:0/1647531976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f867415ec40 0x7f86740d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f867c000640 1 -- 172.21.15.150:0/1647531976 >> 172.21.15.150:0/1647531976 conn(0x7f8674073690 msgr2=0x7f867406cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f5f15e00640 1 -- 172.21.15.150:0/2735797080 wait complete. 2024-04-15T15:24:02.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f867c000640 1 -- 172.21.15.150:0/1647531976 shutdown_connections 2024-04-15T15:24:02.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8034040 con 0x7f9c080d1300 2024-04-15T15:24:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf80443f0 con 0x7f9c080d1300 2024-04-15T15:24:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf80447e0 con 0x7f9c080d1300 2024-04-15T15:24:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f867c000640 1 -- 172.21.15.150:0/1647531976 wait complete. 2024-04-15T15:24:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2693078025 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0815f020 con 0x7f9c080d1300 2024-04-15T15:24:02.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f53e3e00640 1 -- 172.21.15.150:0/311135257 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc02ca80 con 0x7f53e4162e90 2024-04-15T15:24:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f53e3e00640 1 -- 172.21.15.150:0/311135257 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc02cc20 con 0x7f53e4162e90 2024-04-15T15:24:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.010+0000 7f53e3e00640 1 -- 172.21.15.150:0/311135257 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0343f0 con 0x7f53e4162e90 2024-04-15T15:24:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2693078025 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0815f5e0 con 0x7f9c080d1300 2024-04-15T15:24:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f0bbca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 0x7f0bb00c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f0bbca00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bb0053cb0 0x7f0bb00e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f0bbca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 0x7f0bb0052740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f0bbca00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb00e3540 con 0x7f0bb00e3160 2024-04-15T15:24:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f0bbca00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0006640 con 0x7f0bb0053cb0 2024-04-15T15:24:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f0bbca00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb0006b60 con 0x7f0bb003c0a0 2024-04-15T15:24:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 <== osd.0 v2:172.21.15.150:6824/2496520552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc78098e50 con 0x7fcc58074be0 2024-04-15T15:24:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f53ec600640 1 -- 172.21.15.150:0/311135257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4162e90 msgr2=0x7f53e40b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f53ec600640 1 --2- 172.21.15.150:0/311135257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4162e90 0x7f53e40b58e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009e50 tx=0x7f53dc02bee0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.011+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a04038020 con 0x7f3a140d1a50 2024-04-15T15:24:02.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2693078025 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c08068c80 con 0x7f9c080d1300 2024-04-15T15:24:02.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f53ec600640 1 -- 172.21.15.150:0/311135257 shutdown_connections 2024-04-15T15:24:02.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f53ec600640 1 --2- 172.21.15.150:0/311135257 >> v2:172.21.15.150:3300/0 conn(0x7f53e40afd70 0x7f53e4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f53ec600640 1 --2- 172.21.15.150:0/311135257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4162e90 0x7f53e40b58e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f53ec600640 1 -- 172.21.15.150:0/311135257 >> 172.21.15.150:0/311135257 conn(0x7f53e40b2460 msgr2=0x7f53e40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f5f15e00640 1 Processor -- start 2024-04-15T15:24:02.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f53ec600640 1 -- 172.21.15.150:0/311135257 shutdown_connections 2024-04-15T15:24:02.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f0bb7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 0x7f0bb0052740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f0bb7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 0x7f0bb0052740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44526/0 (socket says 172.21.15.150:44526) 2024-04-15T15:24:02.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f0bb7e00640 1 -- 172.21.15.150:0/3981418308 learned_addr learned my addr 172.21.15.150:0/3981418308 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f0bb6a00640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bb0053cb0 0x7f0bb00e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.013+0000 7f0bb7400640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 0x7f0bb00c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.013+0000 7f3a0b400640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f39f406c920 0x7f39f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.013+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3a040ad560 con 0x7f3a140d1a50 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f3a0b400640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f39f4074b40 0x7f39f4076f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 --> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a04007e80 con 0x7f39f4074b40 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3a040b4050 con 0x7f3a140d1a50 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f5f15e00640 1 -- start start 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f5f15e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10072950 0x7f5f1006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f5f15e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f1015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f5f15e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 0x7f5f1015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f5f15e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5f100cd1e0 con 0x7f5f10072950 2024-04-15T15:24:02.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f5f15e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f100cfba0 con 0x7f5f100b5a20 2024-04-15T15:24:02.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f5f15e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f100cd360 con 0x7f5f100b3e00 2024-04-15T15:24:02.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f0bb7400640 1 -- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bb0053cb0 msgr2=0x7f0bb00e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f0bb7400640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bb0053cb0 0x7f0bb00e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f0bb7400640 1 -- 172.21.15.150:0/3981418308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 msgr2=0x7f0bb0052740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f0bb7400640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bb00e3160 0x7f0bb0052740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f0bb7400640 1 -- 172.21.15.150:0/3981418308 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb004e1f0 con 0x7f0bb003c0a0 2024-04-15T15:24:02.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f3a13e00640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f39f4074b40 0x7f39f4076f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.014+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bf804b060 con 0x7f9c080d1300 2024-04-15T15:24:02.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.012+0000 7f53ec600640 1 -- 172.21.15.150:0/311135257 wait complete. 2024-04-15T15:24:02.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f53ec600640 1 Processor -- start 2024-04-15T15:24:02.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f5f15400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 0x7f5f1015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f3a12a00640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f39f406c920 0x7f39f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f5f0fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f1015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f53ec600640 1 -- start start 2024-04-15T15:24:02.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f9bffe00640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9be806c8f0 0x7f9be806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9bf80addd0 con 0x7f9c080d1300 2024-04-15T15:24:02.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f9bffe00640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f9be8074a90 0x7f9be8076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 --> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bf800af30 con 0x7f9be8074a90 2024-04-15T15:24:02.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9bf80ae240 con 0x7f9c080d1300 2024-04-15T15:24:02.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f5f15400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 0x7f5f1015c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53802/0 (socket says 172.21.15.150:53802) 2024-04-15T15:24:02.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f5f0fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f1015be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:41424/0 (socket says 172.21.15.150:41424) 2024-04-15T15:24:02.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f5f15400640 1 -- 172.21.15.150:0/1154439691 learned_addr learned my addr 172.21.15.150:0/1154439691 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f5f14a00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10072950 0x7f5f1006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.015+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6174038020 con 0x7f6188161fc0 2024-04-15T15:24:02.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f3a13e00640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f39f4074b40 0x7f39f4076f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f0bb7400640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 0x7f0bb00c7ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bac033000 tx=0x7f0bac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f9c07400640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9be806c8f0 0x7f9be806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f9c0ca00640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f9be8074a90 0x7f9be8076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f5f14a00640 1 -- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 msgr2=0x7f5f1015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f5f14a00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f1015be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f5f14a00640 1 -- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 msgr2=0x7f5f1015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f5f14a00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f100b5a20 0x7f5f1015c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.016+0000 7f5f14a00640 1 -- 172.21.15.150:0/1154439691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f100cbe90 con 0x7f5f10072950 2024-04-15T15:24:02.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f5f0fe00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f100b3e00 0x7f5f1015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:24:02.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 <== osd.2 v2:172.21.15.150:6808/1792706494 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a04007e80 con 0x7f39f4074b40 2024-04-15T15:24:02.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f5f14a00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10072950 0x7f5f1006f8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5f100cf1a0 tx=0x7f5f0000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac03a970 con 0x7f0bb003c0a0 2024-04-15T15:24:02.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bac03ab50 con 0x7f0bb003c0a0 2024-04-15T15:24:02.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac045450 con 0x7f0bb003c0a0 2024-04-15T15:24:02.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f617f400640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f616806c8f0 0x7f616806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb00e1de0 con 0x7f0bb003c0a0 2024-04-15T15:24:02.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00017070 con 0x7f5f10072950 2024-04-15T15:24:02.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f000052d0 con 0x7f5f10072950 2024-04-15T15:24:02.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00003850 con 0x7f5f10072950 2024-04-15T15:24:02.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb004e490 con 0x7f0bb003c0a0 2024-04-15T15:24:02.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f867c000640 1 Processor -- start 2024-04-15T15:24:02.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f100cc170 con 0x7f5f10072950 2024-04-15T15:24:02.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1006f0a0 con 0x7f5f10072950 2024-04-15T15:24:02.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f61740add90 con 0x7f6188161fc0 2024-04-15T15:24:02.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f617f400640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f6168074a90 0x7f6168076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 --> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6174046e70 con 0x7f6168074a90 2024-04-15T15:24:02.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f61740ae220 con 0x7f6188161fc0 2024-04-15T15:24:02.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f6187400640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f616806c8f0 0x7f616806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f867c000640 1 -- start start 2024-04-15T15:24:02.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.019+0000 7f3a12a00640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f39f406c920 0x7f39f406ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a140d1930 tx=0x7f3a00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.017+0000 7f9c0ca00640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f9be8074a90 0x7f9be8076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.018+0000 7f9c07400640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9be806c8f0 0x7f9be806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4000f60 tx=0x7f9bf4007a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.019+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0bb0005390 con 0x7f0bb003c0a0 2024-04-15T15:24:02.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.019+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f100755c0 con 0x7f5f10072950 2024-04-15T15:24:02.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.019+0000 7f6187400640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f616806c8f0 0x7f616806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6180000f80 tx=0x7f6180001570 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 <== osd.6 v2:172.21.15.187:6811/455912281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bf800af30 con 0x7f9be8074a90 2024-04-15T15:24:02.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f867c000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86740d1300 0x7f86740ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f867c000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86740d2c30 0x7f867415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f867c000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f867415ec40 0x7f86740c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f867c000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86740d3e40 con 0x7f867415ec40 2024-04-15T15:24:02.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f867c000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86740d3b60 con 0x7f86740d2c30 2024-04-15T15:24:02.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f867c000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86740b63b0 con 0x7f86740d1300 2024-04-15T15:24:02.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f53ec600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4072640 0x7f53e41506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f53ec600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53e40afd70 0x7f53e40abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f6186a00640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f6168074a90 0x7f6168076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f53ec600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e4162e90 0x7f53e40ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f53ec600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40cf7a0 con 0x7f53e4162e90 2024-04-15T15:24:02.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f53ec600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40b41a0 con 0x7f53e40afd70 2024-04-15T15:24:02.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.020+0000 7f53ec600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f53e40cf920 con 0x7f53e4072640 2024-04-15T15:24:02.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f8678c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86740d2c30 0x7f867415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f8678c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86740d2c30 0x7f867415fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53812/0 (socket says 172.21.15.150:53812) 2024-04-15T15:24:02.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f8678c00640 1 -- 172.21.15.150:0/2869462948 learned_addr learned my addr 172.21.15.150:0/2869462948 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f53ea600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e4162e90 0x7f53e40ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f53ea600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e4162e90 0x7f53e40ac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44548/0 (socket says 172.21.15.150:44548) 2024-04-15T15:24:02.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f53ea600640 1 -- 172.21.15.150:0/1241433539 learned_addr learned my addr 172.21.15.150:0/1241433539 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f53e9200640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53e40afd70 0x7f53e40abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f867a000640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f867415ec40 0x7f86740c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.022+0000 7f53e9200640 1 -- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4072640 msgr2=0x7f53e41506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.022+0000 7f53e9200640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53e4072640 0x7f53e41506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.022+0000 7f53e9200640 1 -- 172.21.15.150:0/1241433539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e4162e90 msgr2=0x7f53e40ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.022+0000 7f53e9200640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e4162e90 0x7f53e40ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.022+0000 7f53e9200640 1 -- 172.21.15.150:0/1241433539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e40a97b0 con 0x7f53e40afd70 2024-04-15T15:24:02.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.022+0000 7f6186a00640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f6168074a90 0x7f6168076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f8678c00640 1 -- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86740d1300 msgr2=0x7f86740ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f8678c00640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86740d1300 0x7f86740ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f8678c00640 1 -- 172.21.15.150:0/2869462948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f867415ec40 msgr2=0x7f86740c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f8678c00640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f867415ec40 0x7f86740c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.021+0000 7f8678c00640 1 -- 172.21.15.150:0/2869462948 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8674160710 con 0x7f86740d2c30 2024-04-15T15:24:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.022+0000 7f8678c00640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86740d2c30 0x7f867415fc50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8660009e80 tx=0x7f8660004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.023+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660044020 con 0x7f86740d2c30 2024-04-15T15:24:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.023+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866003a040 con 0x7f86740d2c30 2024-04-15T15:24:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.023+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660039070 con 0x7f86740d2c30 2024-04-15T15:24:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.022+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f0001a070 con 0x7f5f10072950 2024-04-15T15:24:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.023+0000 7f5f0cc00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5ef006c940 0x7f5ef006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.024+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5f000573d0 con 0x7f5f10072950 2024-04-15T15:24:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.023+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86740cbaa0 con 0x7f86740d2c30 2024-04-15T15:24:02.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.024+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86740cc060 con 0x7f86740d2c30 2024-04-15T15:24:02.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.024+0000 7f5f0fe00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5ef006c940 0x7f5ef006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.024+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 <== osd.1 v2:172.21.15.150:6800/2508444218 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6174046e70 con 0x7f6168074a90 2024-04-15T15:24:02.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.024+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bac038020 con 0x7f0bb003c0a0 2024-04-15T15:24:02.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.025+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8674068c80 con 0x7f86740d2c30 2024-04-15T15:24:02.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.024+0000 7f5f0cc00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f5ef0075840 0x7f5ef0077ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.024+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 --> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ef00783b0 con 0x7f5ef0075840 2024-04-15T15:24:02.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.025+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5f00055d60 con 0x7f5f10072950 2024-04-15T15:24:02.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.025+0000 7f5f15400640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f5ef0075840 0x7f5ef0077ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.026+0000 7f0bab400640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0b9806c940 0x7f0b9806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.026+0000 7f53e9200640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53e40afd70 0x7f53e40abfb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c7f0 tx=0x7f53d000ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.026+0000 7f5f0fe00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5ef006c940 0x7f5ef006ee00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800e740 tx=0x7f5f0800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.026+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0bac0adfe0 con 0x7f0bb003c0a0 2024-04-15T15:24:02.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.026+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0015070 con 0x7f53e40afd70 2024-04-15T15:24:02.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.026+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e40a9a90 con 0x7f53e40afd70 2024-04-15T15:24:02.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.026+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e40aa050 con 0x7f53e40afd70 2024-04-15T15:24:02.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.026+0000 7f0bab400640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f0b98074b60 0x7f0b98076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.027+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 --> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bac033ec0 con 0x7f0b98074b60 2024-04-15T15:24:02.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.027+0000 7f0bb6a00640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0b9806c940 0x7f0b9806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.027+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0bac0ae3f0 con 0x7f0bb003c0a0 2024-04-15T15:24:02.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.027+0000 7f0bb7e00640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f0b98074b60 0x7f0b98076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.027+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d0004220 con 0x7f53e40afd70 2024-04-15T15:24:02.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.027+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0003610 con 0x7f53e40afd70 2024-04-15T15:24:02.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.028+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53bc001030 con 0x7f53e40afd70 2024-04-15T15:24:02.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.028+0000 7f0bb6a00640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0b9806c940 0x7f0b9806ee00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0053530 tx=0x7f0ba4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.029+0000 7f5f15400640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f5ef0075840 0x7f5ef0077ce0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.029+0000 7f0bb7e00640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f0b98074b60 0x7f0b98076f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.029+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8660047020 con 0x7f86740d2c30 2024-04-15T15:24:02.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.030+0000 7f8671600640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f865406c940 0x7f865406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.030+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f86600ae8c0 con 0x7f86740d2c30 2024-04-15T15:24:02.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.030+0000 7f8671600640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7f8654074b60 0x7f8654076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.030+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 --> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f866002cbe0 con 0x7f8654074b60 2024-04-15T15:24:02.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.030+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f86600aed20 con 0x7f86740d2c30 2024-04-15T15:24:02.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.031+0000 7f867a000640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7f8654074b60 0x7f8654076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.031+0000 7f8679600640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f865406c940 0x7f865406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.032+0000 7f867a000640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7f8654074b60 0x7f8654076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.032+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53d00039c0 con 0x7f53e40afd70 2024-04-15T15:24:02.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.033+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 <== osd.3 v2:172.21.15.150:6816/3354529328 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f866002cbe0 con 0x7f8654074b60 2024-04-15T15:24:02.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.033+0000 7f8679600640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f865406c940 0x7f865406ee00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f866c009820 tx=0x7f866c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.034+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 <== osd.5 v2:172.21.15.187:6800/2717020412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bac033ec0 con 0x7f0b98074b60 2024-04-15T15:24:02.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.034+0000 7f53e1600640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f53c406c920 0x7f53c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.034+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f53d008c010 con 0x7f53e40afd70 2024-04-15T15:24:02.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.035+0000 7f53e1600640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7f53c4074b40 0x7f53c4076f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.035+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 --> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53c4077610 con 0x7f53c4074b40 2024-04-15T15:24:02.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.035+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f53d008c400 con 0x7f53e40afd70 2024-04-15T15:24:02.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.035+0000 7f53e9c00640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f53c406c920 0x7f53c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.035+0000 7f53ea600640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7f53c4074b40 0x7f53c4076f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.036+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 <== osd.4 v2:172.21.15.187:6809/4034856676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ef00783b0 con 0x7f5ef0075840 2024-04-15T15:24:02.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.036+0000 7f53e9c00640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f53c406c920 0x7f53c406ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53dc035000 tx=0x7f53dc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.038+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 --> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a140cecc0 con 0x7f39f4074b40 2024-04-15T15:24:02.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.037+0000 7f53ea600640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7f53c4074b40 0x7f53c4076f60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.038+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 --> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc7004d150 con 0x7fcc58074be0 2024-04-15T15:24:02.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.038+0000 7f3a0b400640 1 -- 172.21.15.150:0/856068747 <== osd.2 v2:172.21.15.150:6808/1792706494 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3a140cecc0 con 0x7f39f4074b40 2024-04-15T15:24:02.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7fcc75600640 1 -- 172.21.15.150:0/709644321 <== osd.0 v2:172.21.15.150:6824/2496520552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcc7004d150 con 0x7fcc58074be0 2024-04-15T15:24:02.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f39f4074b40 msgr2=0x7f39f4076f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7f3a1a800640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f39f4074b40 0x7f39f4076f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7fcc58074be0 msgr2=0x7fcc58077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7fcc7ea00640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7fcc58074be0 0x7fcc58077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc5806c9c0 msgr2=0x7fcc5806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7fcc7ea00640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc5806c9c0 0x7fcc5806ee80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c036000 tx=0x7fcc6c030000 comp rx=0 tx=0).stop 2024-04-15T15:24:02.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 msgr2=0x7fcc700dd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.039+0000 7fcc7ea00640 1 --2- 172.21.15.150:0/709644321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc700e7710 0x7fcc700dd730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc78075df0 tx=0x7fcc78069540 comp rx=0 tx=0).stop 2024-04-15T15:24:02.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f39f406c920 msgr2=0x7f39f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f3a1a800640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f39f406c920 0x7f39f406ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a140d1930 tx=0x7f3a00009040 comp rx=0 tx=0).stop 2024-04-15T15:24:02.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a140d1a50 msgr2=0x7f3a14150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f3a1a800640 1 --2- 172.21.15.150:0/856068747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a140d1a50 0x7f3a14150fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3a04033000 tx=0x7f3a040046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f3a13e00640 1 -- 172.21.15.150:0/856068747 reap_dead start 2024-04-15T15:24:02.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 shutdown_connections 2024-04-15T15:24:02.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 <== osd.7 v2:172.21.15.187:6808/2186687773 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53c4077610 con 0x7f53c4074b40 2024-04-15T15:24:02.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 >> 172.21.15.150:0/856068747 conn(0x7f3a14073690 msgr2=0x7f3a140d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.040+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 shutdown_connections 2024-04-15T15:24:02.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.041+0000 7f3a1a800640 1 -- 172.21.15.150:0/856068747 wait complete. 2024-04-15T15:24:02.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.042+0000 7fcc7e000640 1 -- 172.21.15.150:0/709644321 reap_dead start 2024-04-15T15:24:02.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.042+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 shutdown_connections 2024-04-15T15:24:02.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.042+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 >> 172.21.15.150:0/709644321 conn(0x7fcc70058ee0 msgr2=0x7fcc700e6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.043+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 shutdown_connections 2024-04-15T15:24:02.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.043+0000 7fcc7ea00640 1 -- 172.21.15.150:0/709644321 wait complete. 2024-04-15T15:24:02.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.056+0000 7f9c0ea00640 1 -- 172.21.15.150:0/2693078025 --> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c080c9460 con 0x7f9be8074a90 2024-04-15T15:24:02.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.057+0000 7f9bffe00640 1 -- 172.21.15.150:0/2693078025 <== osd.6 v2:172.21.15.187:6811/455912281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9c080c9460 con 0x7f9be8074a90 2024-04-15T15:24:02.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.058+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 --> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f61880b4990 con 0x7f6168074a90 2024-04-15T15:24:02.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.058+0000 7f9bfd600640 1 -- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f9be8074a90 msgr2=0x7f9be8076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.058+0000 7f9bfd600640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f9be8074a90 0x7f9be8076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f617f400640 1 -- 172.21.15.150:0/2381920424 <== osd.1 v2:172.21.15.150:6800/2508444218 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f61880b4990 con 0x7f6168074a90 2024-04-15T15:24:02.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f9bfd600640 1 -- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9be806c8f0 msgr2=0x7f9be806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f9bfd600640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9be806c8f0 0x7f9be806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4000f60 tx=0x7f9bf4007a80 comp rx=0 tx=0).stop 2024-04-15T15:24:02.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f9bfd600640 1 -- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c080d1300 msgr2=0x7f9c080cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f9bfd600640 1 --2- 172.21.15.150:0/2693078025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c080d1300 0x7f9c080cca80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80098f0 tx=0x7f9bf80046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f9c0ca00640 1 -- 172.21.15.150:0/2693078025 reap_dead start 2024-04-15T15:24:02.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f6168074a90 msgr2=0x7f6168076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f618e400640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f6168074a90 0x7f6168076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f9bfd600640 1 -- 172.21.15.150:0/2693078025 shutdown_connections 2024-04-15T15:24:02.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.059+0000 7f9bfd600640 1 -- 172.21.15.150:0/2693078025 >> 172.21.15.150:0/2693078025 conn(0x7f9c08073690 msgr2=0x7f9c080b9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f616806c8f0 msgr2=0x7f616806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f618e400640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f616806c8f0 0x7f616806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6180000f80 tx=0x7f6180001570 comp rx=0 tx=0).stop 2024-04-15T15:24:02.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6188161fc0 msgr2=0x7f61881f7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f618e400640 1 --2- 172.21.15.150:0/2381920424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6188161fc0 0x7f61881f7870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f61740097a0 tx=0x7f61740046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f6187e00640 1 -- 172.21.15.150:0/2381920424 reap_dead start 2024-04-15T15:24:02.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 shutdown_connections 2024-04-15T15:24:02.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 >> 172.21.15.150:0/2381920424 conn(0x7f6188073480 msgr2=0x7f618815e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f9bfd600640 1 -- 172.21.15.150:0/2693078025 shutdown_connections 2024-04-15T15:24:02.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.061+0000 7f9bfd600640 1 -- 172.21.15.150:0/2693078025 wait complete. 2024-04-15T15:24:02.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.062+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 shutdown_connections 2024-04-15T15:24:02.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.062+0000 7f618e400640 1 -- 172.21.15.150:0/2381920424 wait complete. 2024-04-15T15:24:02.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.062+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 --> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86740d4680 con 0x7f8654074b60 2024-04-15T15:24:02.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.064+0000 7f8671600640 1 -- 172.21.15.150:0/2869462948 <== osd.3 v2:172.21.15.150:6816/3354529328 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f86740d4680 con 0x7f8654074b60 2024-04-15T15:24:02.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.065+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7f8654074b60 msgr2=0x7f8654076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.065+0000 7f867c000640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7f8654074b60 0x7f8654076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.091 INFO:teuthology.orchestra.run.smithi150.stdout:60129542149 2024-04-15T15:24:02.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.065+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f865406c940 msgr2=0x7f865406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.065+0000 7f867c000640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f865406c940 0x7f865406ee00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f866c009820 tx=0x7f866c008040 comp rx=0 tx=0).stop 2024-04-15T15:24:02.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.065+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86740d2c30 msgr2=0x7f867415fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.065+0000 7f867c000640 1 --2- 172.21.15.150:0/2869462948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86740d2c30 0x7f867415fc50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8660009e80 tx=0x7f8660004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:02.093 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-15T15:24:02.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.067+0000 7f867a000640 1 -- 172.21.15.150:0/2869462948 reap_dead start 2024-04-15T15:24:02.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.067+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 shutdown_connections 2024-04-15T15:24:02.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.067+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 >> 172.21.15.150:0/2869462948 conn(0x7f8674073690 msgr2=0x7f86740b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.067+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 --> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5f100707f0 con 0x7f5ef0075840 2024-04-15T15:24:02.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.068+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 shutdown_connections 2024-04-15T15:24:02.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.068+0000 7f867c000640 1 -- 172.21.15.150:0/2869462948 wait complete. 2024-04-15T15:24:02.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.068+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 --> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53bc002dd0 con 0x7f53c4074b40 2024-04-15T15:24:02.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.069+0000 7f5f0cc00640 1 -- 172.21.15.150:0/1154439691 <== osd.4 v2:172.21.15.187:6809/4034856676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5f100707f0 con 0x7f5ef0075840 2024-04-15T15:24:02.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.069+0000 7f53e1600640 1 -- 172.21.15.150:0/1241433539 <== osd.7 v2:172.21.15.187:6808/2186687773 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f53bc002dd0 con 0x7f53c4074b40 2024-04-15T15:24:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.069+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f5ef0075840 msgr2=0x7f5ef0077ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.069+0000 7f5f15e00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f5ef0075840 0x7f5ef0077ce0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.069+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5ef006c940 msgr2=0x7f5ef006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.069+0000 7f5f15e00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5ef006c940 0x7f5ef006ee00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800e740 tx=0x7f5f0800b040 comp rx=0 tx=0).stop 2024-04-15T15:24:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.069+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10072950 msgr2=0x7f5f1006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.069+0000 7f5f15e00640 1 --2- 172.21.15.150:0/1154439691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10072950 0x7f5f1006f8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5f100cf1a0 tx=0x7f5f0000c820 comp rx=0 tx=0).stop 2024-04-15T15:24:02.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.070+0000 7f5f15400640 1 -- 172.21.15.150:0/1154439691 reap_dead start 2024-04-15T15:24:02.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.070+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 shutdown_connections 2024-04-15T15:24:02.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.070+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 >> 172.21.15.150:0/1154439691 conn(0x7f5f10071e30 msgr2=0x7f5f1014b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.070+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 shutdown_connections 2024-04-15T15:24:02.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.070+0000 7f5f15e00640 1 -- 172.21.15.150:0/1154439691 wait complete. 2024-04-15T15:24:02.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.070+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7f53c4074b40 msgr2=0x7f53c4076f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.070+0000 7f53ec600640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7f53c4074b40 0x7f53c4076f60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.098 INFO:teuthology.orchestra.run.smithi150.stdout:64424509446 2024-04-15T15:24:02.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-15T15:24:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f53c406c920 msgr2=0x7f53c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ec600640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f53c406c920 0x7f53c406ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53dc035000 tx=0x7f53dc02e000 comp rx=0 tx=0).stop 2024-04-15T15:24:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53e40afd70 msgr2=0x7f53e40abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ec600640 1 --2- 172.21.15.150:0/1241433539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53e40afd70 0x7f53e40abfb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c7f0 tx=0x7f53d000ccb0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ea600640 1 -- 172.21.15.150:0/1241433539 reap_dead start 2024-04-15T15:24:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 shutdown_connections 2024-04-15T15:24:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 >> 172.21.15.150:0/1241433539 conn(0x7f53e40b2460 msgr2=0x7f53e40ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 --> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0bb00c8530 con 0x7f0b98074b60 2024-04-15T15:24:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 shutdown_connections 2024-04-15T15:24:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.071+0000 7f53ec600640 1 -- 172.21.15.150:0/1241433539 wait complete. 2024-04-15T15:24:02.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.072+0000 7f0bab400640 1 -- 172.21.15.150:0/3981418308 <== osd.5 v2:172.21.15.187:6800/2717020412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0bb00c8530 con 0x7f0b98074b60 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f0b98074b60 msgr2=0x7f0b98076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bbca00640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f0b98074b60 0x7f0b98076f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0b9806c940 msgr2=0x7f0b9806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bbca00640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0b9806c940 0x7f0b9806ee00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0053530 tx=0x7f0ba4009190 comp rx=0 tx=0).stop 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 msgr2=0x7f0bb00c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bbca00640 1 --2- 172.21.15.150:0/3981418308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bb003c0a0 0x7f0bb00c7ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bac033000 tx=0x7f0bac0046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bb7e00640 1 -- 172.21.15.150:0/3981418308 reap_dead start 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 shutdown_connections 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.073+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 >> 172.21.15.150:0/3981418308 conn(0x7f0bb0058ee0 msgr2=0x7f0bb0039d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.074+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 shutdown_connections 2024-04-15T15:24:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.074+0000 7f0bbca00640 1 -- 172.21.15.150:0/3981418308 wait complete. 2024-04-15T15:24:02.103 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:02.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-15T15:24:02.104 INFO:teuthology.orchestra.run.smithi150.stdout:64424509446 2024-04-15T15:24:02.104 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-15T15:24:02.105 INFO:teuthology.orchestra.run.smithi150.stdout:64424509446 2024-04-15T15:24:02.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-15T15:24:02.106 INFO:teuthology.orchestra.run.smithi150.stdout:64424509446 2024-04-15T15:24:02.107 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-15T15:24:02.107 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:02.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-15T15:24:02.108 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:02.109 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-15T15:24:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.426+0000 7fdf8a200640 1 Processor -- start 2024-04-15T15:24:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.427+0000 7fdf8a200640 1 -- start start 2024-04-15T15:24:02.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.429+0000 7fdf88e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fdf7c0e7710 0x7fdf7c04ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33472/0 (socket says 172.21.15.150:33472) 2024-04-15T15:24:02.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.429+0000 7fdf88e00640 1 -- 172.21.15.150:0/218926649 learned_addr learned my addr 172.21.15.150:0/218926649 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:02.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.430+0000 7fdf8a200640 1 --2- 172.21.15.150:0/218926649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c053c40 0x7fdf7c007020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.430+0000 7fdf8a200640 1 -- 172.21.15.150:0/218926649 --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c057d90 con 0x7fdf7c0e7710 2024-04-15T15:24:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.430+0000 7fdf8a200640 1 -- 172.21.15.150:0/218926649 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c006260 con 0x7fdf7c0589c0 2024-04-15T15:24:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.430+0000 7fdf8a200640 1 -- 172.21.15.150:0/218926649 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c006540 con 0x7fdf7c053c40 2024-04-15T15:24:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.431+0000 7fdf89800640 1 --2- 172.21.15.150:0/218926649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c053c40 0x7fdf7c007020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.431+0000 7fe050e00640 1 Processor -- start 2024-04-15T15:24:02.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.431+0000 7fdf83400640 1 -- 172.21.15.150:0/218926649 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 519010668 0 0) 0x7fdf7c057d90 con 0x7fdf7c0e7710 2024-04-15T15:24:02.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.431+0000 7fdf83400640 1 -- 172.21.15.150:0/218926649 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf6c003520 con 0x7fdf7c0e7710 2024-04-15T15:24:02.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.432+0000 7fdf83400640 1 -- 172.21.15.150:0/218926649 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073536774 0 0) 0x7fdf7c006260 con 0x7fdf7c0589c0 2024-04-15T15:24:02.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.432+0000 7fdf83400640 1 -- 172.21.15.150:0/218926649 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf7c057d90 con 0x7fdf7c0589c0 2024-04-15T15:24:02.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.432+0000 7fdf89800640 1 -- 172.21.15.150:0/218926649 >> v1:172.21.15.187:6789/0 conn(0x7fdf7c0589c0 legacy=0x7fdf7c053530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.432+0000 7fdf89800640 1 -- 172.21.15.150:0/218926649 >> v1:172.21.15.150:6789/0 conn(0x7fdf7c0e7710 legacy=0x7fdf7c04ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.432+0000 7fdf89800640 1 -- 172.21.15.150:0/218926649 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c006820 con 0x7fdf7c053c40 2024-04-15T15:24:02.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.432+0000 7f09b9800640 1 Processor -- start 2024-04-15T15:24:02.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.431+0000 7fe050e00640 1 -- start start 2024-04-15T15:24:02.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7fe050e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe044048100 0x7fe044006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7fe050e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe044007460 0x7fe044053e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7fdf89800640 1 --2- 172.21.15.150:0/218926649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c053c40 0x7fdf7c007020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf700097d0 tx=0x7fdf7002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3c52ca85d15591 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7fdf83400640 1 -- 172.21.15.150:0/218926649 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70034070 con 0x7fdf7c053c40 2024-04-15T15:24:02.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.433+0000 7fc9d8600640 1 Processor -- start 2024-04-15T15:24:02.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.433+0000 7fc9d8600640 1 -- start start 2024-04-15T15:24:02.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.432+0000 7f09b9800640 1 -- start start 2024-04-15T15:24:02.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7f09b9800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f09b40b3560 0x7f09b40ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7f09b9800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b40ba6a0 con 0x7f09b40b6d30 2024-04-15T15:24:02.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7f09b9800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b40cf8f0 con 0x7f09b415eb50 2024-04-15T15:24:02.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7f09b9800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f09b415ef30 con 0x7f09b40b3560 2024-04-15T15:24:02.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.434+0000 7fe050e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe044005ec0 con 0x7fe044054390 2024-04-15T15:24:02.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fe049e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe044048100 0x7fe044006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fe04a800640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fe044054390 0x7fe04404f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:51488/0 (socket says 172.21.15.150:51488) 2024-04-15T15:24:02.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fe049e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe044048100 0x7fe044006f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53840/0 (socket says 172.21.15.150:53840) 2024-04-15T15:24:02.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fe049400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe044007460 0x7fe044053e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fe04a800640 1 -- 172.21.15.150:0/2230851008 learned_addr learned my addr 172.21.15.150:0/2230851008 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:02.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fe050e00640 1 -- 172.21.15.150:0/2230851008 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe0440e7b90 con 0x7fe044048100 2024-04-15T15:24:02.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fe050e00640 1 -- 172.21.15.150:0/2230851008 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0440547e0 con 0x7fe044007460 2024-04-15T15:24:02.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fdf83400640 1 -- 172.21.15.150:0/218926649 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf70030800 con 0x7fdf7c053c40 2024-04-15T15:24:02.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7f09b8e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f09b40b3560 0x7f09b40ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7f09b8e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f09b40b3560 0x7f09b40ba0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53848/0 (socket says 172.21.15.150:53848) 2024-04-15T15:24:02.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7f09b3400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f09b40b6d30 0x7f09b406ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33478/0 (socket says 172.21.15.150:33478) 2024-04-15T15:24:02.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7f09b3400640 1 -- 172.21.15.150:0/2074367692 learned_addr learned my addr 172.21.15.150:0/2074367692 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:02.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7f09b3e00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f09b415eb50 0x7f09b40b6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:51502/0 (socket says 172.21.15.150:51502) 2024-04-15T15:24:02.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7fc9d8600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc9d0072810 0x7fc9d00b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7fc9d8600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9d00b01a0 con 0x7fc9d0162e90 2024-04-15T15:24:02.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7fc9d8600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9d00b3900 con 0x7fc9d00afd70 2024-04-15T15:24:02.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7fc9d8600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d00b3be0 con 0x7fc9d0072810 2024-04-15T15:24:02.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 502661626 0 0) 0x7fe044005ec0 con 0x7fe044054390 2024-04-15T15:24:02.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe030003520 con 0x7fe044054390 2024-04-15T15:24:02.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7f5b82800640 1 Processor -- start 2024-04-15T15:24:02.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.435+0000 7fdf83400640 1 -- 172.21.15.150:0/218926649 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70030bb0 con 0x7fdf7c053c40 2024-04-15T15:24:02.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7fc9d5c00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc9d0162e90 0x7fc9d0072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33490/0 (socket says 172.21.15.150:33490) 2024-04-15T15:24:02.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.436+0000 7fc9d5c00640 1 -- 172.21.15.150:0/2943260204 learned_addr learned my addr 172.21.15.150:0/2943260204 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:02.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 892768790 0 0) 0x7fe030003520 con 0x7fe044054390 2024-04-15T15:24:02.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fc9d5200640 1 --2- 172.21.15.150:0/2943260204 >> v2:172.21.15.150:3300/0 conn(0x7fc9d0072810 0x7fc9d00b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe044007460 msgr2=0x7fe044053e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe044007460 0x7fe044053e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 >> v2:172.21.15.187:3300/0 conn(0x7fe044048100 msgr2=0x7fe044006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 --2- 172.21.15.150:0/2230851008 >> v2:172.21.15.187:3300/0 conn(0x7fe044048100 0x7fe044006f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe044053280 con 0x7fe044054390 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fe03c001670 con 0x7fe044054390 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 >> v1:172.21.15.187:6789/0 conn(0x7fe044054390 legacy=0x7fe04404f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0300039b0 0x7fe030003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe030004800 0x7fe030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0300051c0 0x7fe03000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440e7b90 con 0x7fe0300039b0 2024-04-15T15:24:02.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe03000bf00 con 0x7fe0300051c0 2024-04-15T15:24:02.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe03000c080 con 0x7fe030004800 2024-04-15T15:24:02.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615991074 0 0) 0x7fc9d00b01a0 con 0x7fc9d0162e90 2024-04-15T15:24:02.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9b8003520 con 0x7fc9d0162e90 2024-04-15T15:24:02.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823913030 0 0) 0x7fc9d00b3900 con 0x7fc9d00afd70 2024-04-15T15:24:02.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9d00b01a0 con 0x7fc9d00afd70 2024-04-15T15:24:02.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fdf8a200640 1 -- 172.21.15.150:0/218926649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c053c40 msgr2=0x7fdf7c007020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fdf8a200640 1 --2- 172.21.15.150:0/218926649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c053c40 0x7fdf7c007020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf700097d0 tx=0x7fdf7002b940 comp rx=0 tx=0).stop 2024-04-15T15:24:02.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7f09b2a00640 1 -- 172.21.15.150:0/2074367692 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 697910450 0 0) 0x7f09b40cf8f0 con 0x7f09b415eb50 2024-04-15T15:24:02.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fe049400640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe030004800 0x7fe030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7f09b2a00640 1 -- 172.21.15.150:0/2074367692 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0998003520 con 0x7f09b415eb50 2024-04-15T15:24:02.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7f09b2a00640 1 -- 172.21.15.150:0/2074367692 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1506303557 0 0) 0x7f09b40ba6a0 con 0x7f09b40b6d30 2024-04-15T15:24:02.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7f09b2a00640 1 -- 172.21.15.150:0/2074367692 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09b40cf8f0 con 0x7f09b40b6d30 2024-04-15T15:24:02.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fdf8a200640 1 -- 172.21.15.150:0/218926649 shutdown_connections 2024-04-15T15:24:02.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fdf8a200640 1 --2- 172.21.15.150:0/218926649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c053c40 0x7fdf7c007020 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fdf8a200640 1 -- 172.21.15.150:0/218926649 >> 172.21.15.150:0/218926649 conn(0x7fdf7c058ee0 msgr2=0x7fdf7c0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2928689554 0 0) 0x7fc9d00b01a0 con 0x7fc9d00afd70 2024-04-15T15:24:02.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 >> v2:172.21.15.150:3300/0 conn(0x7fc9d0072810 msgr2=0x7fc9d00b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fc9cfe00640 1 --2- 172.21.15.150:0/2943260204 >> v2:172.21.15.150:3300/0 conn(0x7fc9d0072810 0x7fc9d00b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.438+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 >> v1:172.21.15.150:6789/0 conn(0x7fc9d0162e90 legacy=0x7fc9d0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.437+0000 7f5b82800640 1 -- start start 2024-04-15T15:24:02.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fe049e00640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0300039b0 0x7fe030003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fe04a800640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0300051c0 0x7fe03000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7f5b82800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b6d30 0x7f5b7c0ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7f5b82800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5b7c0b3560 0x7f5b7c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7f5b82800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7c0ba6a0 con 0x7f5b7c15eb50 2024-04-15T15:24:02.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7f5b82800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5b7c15ef30 con 0x7f5b7c0b3560 2024-04-15T15:24:02.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7f5b82800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7c0cf6a0 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fe049400640 1 -- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0300051c0 msgr2=0x7fe03000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fe049400640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0300051c0 0x7fe03000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fe049400640 1 -- 172.21.15.150:0/2230851008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0300039b0 msgr2=0x7fe030003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fdf8a200640 1 -- 172.21.15.150:0/218926649 shutdown_connections 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fdf8a200640 1 -- 172.21.15.150:0/218926649 wait complete. 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d00b3ec0 con 0x7fc9d00afd70 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fc9c8001670 con 0x7fc9d00afd70 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 >> v1:172.21.15.187:6789/0 conn(0x7fc9d00afd70 legacy=0x7fc9d00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b8004ba0 0x7fc9b8004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b80037a0 0x7fc9b8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9b8004160 0x7fc9b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d00b3be0 con 0x7fc9b8004160 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b800c060 con 0x7fc9b8004ba0 2024-04-15T15:24:02.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b800c260 con 0x7fc9b80037a0 2024-04-15T15:24:02.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fc9d6600640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9b8004160 0x7fc9b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fc9d5c00640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b8004ba0 0x7fc9b8004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7f09b8e00640 1 -- 172.21.15.150:0/2074367692 >> v1:172.21.15.187:6789/0 conn(0x7f09b415eb50 legacy=0x7f09b40b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7f5b81e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5b7c0b3560 0x7f5b7c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7f5b81e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5b7c0b3560 0x7f5b7c0b78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53856/0 (socket says 172.21.15.150:53856) 2024-04-15T15:24:02.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7f5b81e00640 1 -- 172.21.15.150:0/356948942 learned_addr learned my addr 172.21.15.150:0/356948942 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fe049400640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0300039b0 0x7fe030003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fe049400640 1 -- 172.21.15.150:0/2230851008 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe044053280 con 0x7fe030004800 2024-04-15T15:24:02.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7f5b80a00640 1 --2- 172.21.15.150:0/356948942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b6d30 0x7f5b7c0ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fc9d6600640 1 -- 172.21.15.150:0/2943260204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b80037a0 msgr2=0x7fc9b8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fc9d6600640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b80037a0 0x7fc9b8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fc9d6600640 1 -- 172.21.15.150:0/2943260204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b8004ba0 msgr2=0x7fc9b8004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fc9d6600640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9b8004ba0 0x7fc9b8004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7fc9d6600640 1 -- 172.21.15.150:0/2943260204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d00b3ec0 con 0x7fc9b8004160 2024-04-15T15:24:02.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7f09b8e00640 1 -- 172.21.15.150:0/2074367692 >> v1:172.21.15.150:6789/0 conn(0x7f09b40b6d30 legacy=0x7f09b406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.439+0000 7f09b8e00640 1 -- 172.21.15.150:0/2074367692 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b40cfbd0 con 0x7f09b40b3560 2024-04-15T15:24:02.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.441+0000 7fe049e00640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0300039b0 0x7fe030003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:02.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.441+0000 7f09b8e00640 1 --2- 172.21.15.150:0/2074367692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b3560 0x7f09b40ba0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009d90 tx=0x7f09ac02bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4923f8dd8cc09a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.440+0000 7ff1cce00640 1 Processor -- start 2024-04-15T15:24:02.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.441+0000 7ff1cce00640 1 -- start start 2024-04-15T15:24:02.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.442+0000 7f09b2a00640 1 -- 172.21.15.150:0/2074367692 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac039070 con 0x7f09b40b3560 2024-04-15T15:24:02.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.442+0000 7fc9d6600640 1 -- 172.21.15.150:0/2943260204 reap_dead start 2024-04-15T15:24:02.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.442+0000 7f09b2a00640 1 -- 172.21.15.150:0/2074367692 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09ac02ccb0 con 0x7f09b40b3560 2024-04-15T15:24:02.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.442+0000 7fe049400640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe030004800 0x7fe030004c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe038009980 tx=0x7fe030005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd33c7b3705e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.441+0000 7f5b81e00640 1 -- 172.21.15.150:0/356948942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b6d30 msgr2=0x7f5b7c0ba0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.441+0000 7f5b81e00640 1 --2- 172.21.15.150:0/356948942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b6d30 0x7f5b7c0ba0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.441+0000 7f5b81e00640 1 -- 172.21.15.150:0/356948942 >> v1:172.21.15.187:6789/0 conn(0x7f5b7c15eb50 legacy=0x7f5b7c0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.441+0000 7f5b81e00640 1 -- 172.21.15.150:0/356948942 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c0cf8f0 con 0x7f5b7c0b3560 2024-04-15T15:24:02.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.442+0000 7f09b2a00640 1 -- 172.21.15.150:0/2074367692 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac02ce60 con 0x7f09b40b3560 2024-04-15T15:24:02.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.442+0000 7fc9d6600640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9b8004160 0x7fc9b800b9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800c190 tx=0x7fc9b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b409addf0147b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.442+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038039070 con 0x7fe030004800 2024-04-15T15:24:02.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8039070 con 0x7fc9b8004160 2024-04-15T15:24:02.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c802c9e0 con 0x7fc9b8004160 2024-04-15T15:24:02.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7fdf8a200640 1 Processor -- start 2024-04-15T15:24:02.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7fc9cfe00640 1 -- 172.21.15.150:0/2943260204 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8007c40 con 0x7fc9b8004160 2024-04-15T15:24:02.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7fdf8a200640 1 -- start start 2024-04-15T15:24:02.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7f09b9800640 1 -- 172.21.15.150:0/2074367692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b3560 msgr2=0x7f09b40ba0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7f09b9800640 1 --2- 172.21.15.150:0/2074367692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b3560 0x7f09b40ba0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009d90 tx=0x7f09ac02bf00 comp rx=0 tx=0).stop 2024-04-15T15:24:02.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7f09b9800640 1 -- 172.21.15.150:0/2074367692 shutdown_connections 2024-04-15T15:24:02.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7f09b9800640 1 --2- 172.21.15.150:0/2074367692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b3560 0x7f09b40ba0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7f09b9800640 1 -- 172.21.15.150:0/2074367692 >> 172.21.15.150:0/2074367692 conn(0x7f09b40d1700 msgr2=0x7f09b40d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7ff1cce00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff1c80b3760 0x7ff1c80b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7ff1cce00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c80b44c0 con 0x7ff1c8072100 2024-04-15T15:24:02.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7ff1cce00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c80b47a0 con 0x7ff1c80701f0 2024-04-15T15:24:02.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.443+0000 7ff1cce00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c8161610 con 0x7ff1c80b3760 2024-04-15T15:24:02.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7fc9d8600640 1 -- 172.21.15.150:0/2943260204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9b8004160 msgr2=0x7fc9b800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7fc9d8600640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9b8004160 0x7fc9b800b9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800c190 tx=0x7fc9b8005830 comp rx=0 tx=0).stop 2024-04-15T15:24:02.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03802c580 con 0x7fe030004800 2024-04-15T15:24:02.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7ff1c6800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7ff1c8072100 0x7ff1c80cf0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33494/0 (socket says 172.21.15.150:33494) 2024-04-15T15:24:02.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7ff1c6800640 1 -- 172.21.15.150:0/3628230741 learned_addr learned my addr 172.21.15.150:0/3628230741 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:02.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7ff1c5e00640 1 --1- 172.21.15.150:0/3628230741 >> v1:172.21.15.187:6789/0 conn(0x7ff1c80701f0 0x7ff1c815a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:51516/0 (socket says 172.21.15.150:51516) 2024-04-15T15:24:02.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7fe048a00640 1 -- 172.21.15.150:0/2230851008 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038009650 con 0x7fe030004800 2024-04-15T15:24:02.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7ff1c5400640 1 --2- 172.21.15.150:0/3628230741 >> v2:172.21.15.187:3300/0 conn(0x7ff1c80b3760 0x7ff1c80b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7f09b9800640 1 -- 172.21.15.150:0/2074367692 shutdown_connections 2024-04-15T15:24:02.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7fc9d8600640 1 -- 172.21.15.150:0/2943260204 shutdown_connections 2024-04-15T15:24:02.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7fc9d8600640 1 --2- 172.21.15.150:0/2943260204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9b8004160 0x7fc9b800b9c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.444+0000 7fc9d8600640 1 -- 172.21.15.150:0/2943260204 >> 172.21.15.150:0/2943260204 conn(0x7fc9d00b2460 msgr2=0x7fc9d00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7f09b9800640 1 -- 172.21.15.150:0/2074367692 wait complete. 2024-04-15T15:24:02.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7f5b81e00640 1 --2- 172.21.15.150:0/356948942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b3560 0x7f5b7c0b78d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009870 tx=0x7f5b6802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a859a2848830f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7fdf8a200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf7c053c40 0x7fdf7c0df240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7fdf8a200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c0589c0 0x7fdf7c0d1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7fdf8a200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf7c0e7710 0x7fdf7c03b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7fdf8a200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c057b20 con 0x7fdf7c053c40 2024-04-15T15:24:02.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7fdf8a200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c0579a0 con 0x7fdf7c0e7710 2024-04-15T15:24:02.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7fdf8a200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c056ab0 con 0x7fdf7c0589c0 2024-04-15T15:24:02.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7ff1c5400640 1 -- 172.21.15.150:0/3628230741 >> v1:172.21.15.187:6789/0 conn(0x7ff1c80701f0 legacy=0x7ff1c815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7ff1c5400640 1 -- 172.21.15.150:0/3628230741 >> v1:172.21.15.150:6789/0 conn(0x7ff1c8072100 legacy=0x7ff1c80cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7ff1c5400640 1 -- 172.21.15.150:0/3628230741 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c80b4a80 con 0x7ff1c80b3760 2024-04-15T15:24:02.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7f5b7be00640 1 -- 172.21.15.150:0/356948942 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68035020 con 0x7f5b7c0b3560 2024-04-15T15:24:02.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf88e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf7c053c40 0x7fdf7c0df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf88e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf7c053c40 0x7fdf7c0df240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44600/0 (socket says 172.21.15.150:44600) 2024-04-15T15:24:02.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf88e00640 1 -- 172.21.15.150:0/1237486247 learned_addr learned my addr 172.21.15.150:0/1237486247 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf83e00640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c0589c0 0x7fdf7c0d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf89800640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf7c0e7710 0x7fdf7c03b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.447+0000 7f5b7be00640 1 -- 172.21.15.150:0/356948942 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68031640 con 0x7f5b7c0b3560 2024-04-15T15:24:02.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.447+0000 7f5b7be00640 1 -- 172.21.15.150:0/356948942 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6802d040 con 0x7f5b7c0b3560 2024-04-15T15:24:02.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.447+0000 7fe050e00640 1 -- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe030004800 msgr2=0x7fe030004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.447+0000 7fe050e00640 1 --2- 172.21.15.150:0/2230851008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe030004800 0x7fe030004c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe038009980 tx=0x7fe030005830 comp rx=0 tx=0).stop 2024-04-15T15:24:02.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.447+0000 7f5b82800640 1 -- 172.21.15.150:0/356948942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b3560 msgr2=0x7f5b7c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7f5b82800640 1 --2- 172.21.15.150:0/356948942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b3560 0x7f5b7c0b78d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009870 tx=0x7f5b6802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf88e00640 1 -- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c0589c0 msgr2=0x7fdf7c0d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf88e00640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c0589c0 0x7fdf7c0d1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf88e00640 1 -- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf7c0e7710 msgr2=0x7fdf7c03b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf88e00640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf7c0e7710 0x7fdf7c03b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.446+0000 7fdf88e00640 1 -- 172.21.15.150:0/1237486247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0df780 con 0x7fdf7c053c40 2024-04-15T15:24:02.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fdf88e00640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf7c053c40 0x7fdf7c0df240 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c430 tx=0x7fdf7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7fc9d8600640 1 -- 172.21.15.150:0/2943260204 shutdown_connections 2024-04-15T15:24:02.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.445+0000 7fc9d8600640 1 -- 172.21.15.150:0/2943260204 wait complete. 2024-04-15T15:24:02.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.447+0000 7fc9d8600640 1 Processor -- start 2024-04-15T15:24:02.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.447+0000 7fc9d8600640 1 -- start start 2024-04-15T15:24:02.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b8004160 0x7fc9d00af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d8600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9d015e0e0 0x7fc9d01654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d8600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9d00a94a0 0x7fc9d00a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d8600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d00b5a60 con 0x7fc9d015e0e0 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d8600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d00b58e0 con 0x7fc9d00a94a0 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d00b5be0 con 0x7fc9b8004160 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d5200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9d015e0e0 0x7fc9d01654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d5200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9d015e0e0 0x7fc9d01654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44604/0 (socket says 172.21.15.150:44604) 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.448+0000 7fc9d5200640 1 -- 172.21.15.150:0/2606201826 learned_addr learned my addr 172.21.15.150:0/2606201826 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fc9d5c00640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b8004160 0x7fc9d00af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fe04a800640 1 -- 172.21.15.150:0/2230851008 reap_dead start 2024-04-15T15:24:02.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fe050e00640 1 -- 172.21.15.150:0/2230851008 shutdown_connections 2024-04-15T15:24:02.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fe050e00640 1 -- 172.21.15.150:0/2230851008 >> 172.21.15.150:0/2230851008 conn(0x7fe044009fe0 msgr2=0x7fe04400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fc9d6600640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9d00a94a0 0x7fc9d00a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fe050e00640 1 -- 172.21.15.150:0/2230851008 shutdown_connections 2024-04-15T15:24:02.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fe050e00640 1 -- 172.21.15.150:0/2230851008 wait complete. 2024-04-15T15:24:02.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7ff1c5400640 1 --2- 172.21.15.150:0/3628230741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 0x7ff1c80b3bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40097a0 tx=0x7ff1b402b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=115866f4c03d7544 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fc9d5200640 1 -- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b8004160 msgr2=0x7fc9d00af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fc9d5200640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b8004160 0x7fc9d00af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fc9d5200640 1 -- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9d00a94a0 msgr2=0x7fc9d00a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fc9d5200640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9d00a94a0 0x7fc9d00a9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fc9d5200640 1 -- 172.21.15.150:0/2606201826 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d015ed40 con 0x7fc9d015e0e0 2024-04-15T15:24:02.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7f5b82800640 1 -- 172.21.15.150:0/356948942 shutdown_connections 2024-04-15T15:24:02.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7f5b82800640 1 --2- 172.21.15.150:0/356948942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b3560 0x7f5b7c0b78d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7f5b82800640 1 --2- 172.21.15.150:0/356948942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b6d30 0x7f5b7c0ba0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7f5b82800640 1 -- 172.21.15.150:0/356948942 >> 172.21.15.150:0/356948942 conn(0x7f5b7c0d1700 msgr2=0x7f5b7c0d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7f09b9800640 1 Processor -- start 2024-04-15T15:24:02.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fdf80c00640 1 -- 172.21.15.150:0/1237486247 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78017070 con 0x7fdf7c053c40 2024-04-15T15:24:02.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.449+0000 7fdf80c00640 1 -- 172.21.15.150:0/1237486247 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf78005ce0 con 0x7fdf7c053c40 2024-04-15T15:24:02.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7fdf80c00640 1 -- 172.21.15.150:0/1237486247 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78007310 con 0x7fdf7c053c40 2024-04-15T15:24:02.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7fc9d5c00640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b8004160 0x7fc9d00af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:02.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7f5b82800640 1 -- 172.21.15.150:0/356948942 shutdown_connections 2024-04-15T15:24:02.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7ff1c4a00640 1 -- 172.21.15.150:0/3628230741 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4037070 con 0x7ff1c80b3760 2024-04-15T15:24:02.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7ff1c4a00640 1 -- 172.21.15.150:0/3628230741 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b4030710 con 0x7ff1c80b3760 2024-04-15T15:24:02.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7ff1c4a00640 1 -- 172.21.15.150:0/3628230741 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4030ae0 con 0x7ff1c80b3760 2024-04-15T15:24:02.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7f5b82800640 1 -- 172.21.15.150:0/356948942 wait complete. 2024-04-15T15:24:02.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7fdf8a200640 1 -- 172.21.15.150:0/1237486247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c04cb40 con 0x7fdf7c053c40 2024-04-15T15:24:02.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7fdf8a200640 1 -- 172.21.15.150:0/1237486247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c04d100 con 0x7fdf7c053c40 2024-04-15T15:24:02.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7fc9d5200640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9d015e0e0 0x7fc9d01654f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00e7d0 tx=0x7fc9bc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.450+0000 7f09b9800640 1 -- start start 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.451+0000 7f09b9800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09b40b3560 0x7f09b40ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.451+0000 7f09b9800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b6d30 0x7f09b40cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.451+0000 7f09b9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b415eb50 0x7f09b41585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.451+0000 7f09b9800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09b40b3cd0 con 0x7f09b40b3560 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.451+0000 7f09b9800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f09b40cf6a0 con 0x7f09b40b6d30 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.451+0000 7f09b9800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f09b40b78d0 con 0x7f09b415eb50 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b3400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b6d30 0x7f09b40cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b3400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b6d30 0x7f09b40cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53916/0 (socket says 172.21.15.150:53916) 2024-04-15T15:24:02.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b3400640 1 -- 172.21.15.150:0/1569800924 learned_addr learned my addr 172.21.15.150:0/1569800924 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f5b82800640 1 Processor -- start 2024-04-15T15:24:02.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b8e00640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b415eb50 0x7f09b41585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f5b82800640 1 -- start start 2024-04-15T15:24:02.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b3400640 1 -- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b415eb50 msgr2=0x7f09b41585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b3400640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b415eb50 0x7f09b41585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b3400640 1 -- 172.21.15.150:0/1569800924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09b40b3560 msgr2=0x7f09b40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b3400640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09b40b3560 0x7f09b40ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f09b3400640 1 -- 172.21.15.150:0/1569800924 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b40ccab0 con 0x7f09b40b6d30 2024-04-15T15:24:02.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7f4fbca00640 1 Processor -- start 2024-04-15T15:24:02.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7ff1cce00640 1 -- 172.21.15.150:0/3628230741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 msgr2=0x7ff1c80b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.452+0000 7ff1cce00640 1 --2- 172.21.15.150:0/3628230741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 0x7ff1c80b3bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40097a0 tx=0x7ff1b402b680 comp rx=0 tx=0).stop 2024-04-15T15:24:02.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.451+0000 7fe050e00640 1 Processor -- start 2024-04-15T15:24:02.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f4fbca00640 1 -- start start 2024-04-15T15:24:02.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7ff1cce00640 1 -- 172.21.15.150:0/3628230741 shutdown_connections 2024-04-15T15:24:02.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7ff1cce00640 1 --2- 172.21.15.150:0/3628230741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 0x7ff1c80b3bf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f09b3400640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b6d30 0x7f09b40cc340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09a0001eb0 tx=0x7f09a000c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7ff1cce00640 1 -- 172.21.15.150:0/3628230741 >> 172.21.15.150:0/3628230741 conn(0x7ff1c80b2460 msgr2=0x7ff1c80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7fdf76000640 1 -- 172.21.15.150:0/1237486247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf58005290 con 0x7fdf7c053c40 2024-04-15T15:24:02.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f09abe00640 1 -- 172.21.15.150:0/1569800924 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0017070 con 0x7f09b40b6d30 2024-04-15T15:24:02.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7fc9cd600640 1 -- 172.21.15.150:0/2606201826 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc017070 con 0x7fc9d015e0e0 2024-04-15T15:24:02.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f09abe00640 1 -- 172.21.15.150:0/1569800924 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a0005ce0 con 0x7f09b40b6d30 2024-04-15T15:24:02.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f09abe00640 1 -- 172.21.15.150:0/1569800924 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0007290 con 0x7f09b40b6d30 2024-04-15T15:24:02.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d00aa880 con 0x7fc9d015e0e0 2024-04-15T15:24:02.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d00aae40 con 0x7fc9d015e0e0 2024-04-15T15:24:02.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7fc9cd600640 1 -- 172.21.15.150:0/2606201826 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9bc00c940 con 0x7fc9d015e0e0 2024-04-15T15:24:02.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b40ccd90 con 0x7f09b40b6d30 2024-04-15T15:24:02.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b40cd350 con 0x7f09b40b6d30 2024-04-15T15:24:02.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7f4fbca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7fc9cd600640 1 -- 172.21.15.150:0/2606201826 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc00cd10 con 0x7fc9d015e0e0 2024-04-15T15:24:02.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7fe050e00640 1 -- start start 2024-04-15T15:24:02.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7ff1cce00640 1 -- 172.21.15.150:0/3628230741 shutdown_connections 2024-04-15T15:24:02.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f4fbca00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb80d32c0 con 0x7f4fb80d1300 2024-04-15T15:24:02.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f4fbca00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb80d35a0 con 0x7f4fb80d2c30 2024-04-15T15:24:02.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f4fbca00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb80d3090 con 0x7f4fb815ec40 2024-04-15T15:24:02.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f4fb5e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7f4fb5e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:41548/0 (socket says 172.21.15.150:41548) 2024-04-15T15:24:02.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7f4fb5e00640 1 -- 172.21.15.150:0/2661654753 learned_addr learned my addr 172.21.15.150:0/2661654753 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b40755c0 con 0x7f09b40b6d30 2024-04-15T15:24:02.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7f4fb5e00640 1 -- 172.21.15.150:0/2661654753 >> v1:172.21.15.187:6789/0 conn(0x7f4fb80d2c30 legacy=0x7f4fb80ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7f4fb4a00640 1 -- 172.21.15.150:0/2661654753 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358577946 0 0) 0x7f4fb80d35a0 con 0x7f4fb80d2c30 2024-04-15T15:24:02.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7f4fb5e00640 1 -- 172.21.15.150:0/2661654753 >> v1:172.21.15.150:6789/0 conn(0x7f4fb80d1300 legacy=0x7f4fb80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7f4fb5e00640 1 -- 172.21.15.150:0/2661654753 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb80d3880 con 0x7f4fb815ec40 2024-04-15T15:24:02.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.457+0000 7fe050e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0440d11e0 0x7fe044172060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.457+0000 7fe050e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0440d4670 0x7fe044034450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.457+0000 7fe050e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe044034990 0x7fe044036400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f5b82800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b3560 0x7f5b7c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f5b82800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b6d30 0x7f5b7c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f5b82800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b7c15eb50 0x7f5b7c158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f5b82800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c0b3cd0 con 0x7f5b7c15eb50 2024-04-15T15:24:02.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f5b82800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c0cfeb0 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.453+0000 7f5b82800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7c06ced0 con 0x7f5b7c0b3560 2024-04-15T15:24:02.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7f5b80a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b6d30 0x7f5b7c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7f5b80a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b6d30 0x7f5b7c152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53920/0 (socket says 172.21.15.150:53920) 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.454+0000 7f5b80a00640 1 -- 172.21.15.150:0/1179096554 learned_addr learned my addr 172.21.15.150:0/1179096554 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f5b80a00640 1 -- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b3560 msgr2=0x7f5b7c0ca510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f5b80a00640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b3560 0x7f5b7c0ca510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f5b80a00640 1 -- 172.21.15.150:0/1179096554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b7c15eb50 msgr2=0x7f5b7c158a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f5b80a00640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b7c15eb50 0x7f5b7c158a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.455+0000 7f5b80a00640 1 -- 172.21.15.150:0/1179096554 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c0cc9d0 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7f5b80a00640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b6d30 0x7f5b7c152e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000e900 tx=0x7f5b7000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.457+0000 7f5b79600640 1 -- 172.21.15.150:0/1179096554 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70028070 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.457+0000 7f5b79600640 1 -- 172.21.15.150:0/1179096554 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7000ca00 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.457+0000 7f5b79600640 1 -- 172.21.15.150:0/1179096554 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000cdd0 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.457+0000 7fe050e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe044006440 con 0x7fe044034990 2024-04-15T15:24:02.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.458+0000 7fe050e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe044006a90 con 0x7fe0440d11e0 2024-04-15T15:24:02.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.458+0000 7fe050e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe044006720 con 0x7fe0440d4670 2024-04-15T15:24:02.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.458+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c0cccb0 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.458+0000 7fe049e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0440d11e0 0x7fe044172060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.458+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c0cd270 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.458+0000 7fe049e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0440d11e0 0x7fe044172060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53932/0 (socket says 172.21.15.150:53932) 2024-04-15T15:24:02.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.458+0000 7fe049e00640 1 -- 172.21.15.150:0/2866621850 learned_addr learned my addr 172.21.15.150:0/2866621850 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7fe049e00640 1 -- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0440d4670 msgr2=0x7fe044034450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7fe049e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0440d4670 0x7fe044034450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7fe049e00640 1 -- 172.21.15.150:0/2866621850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe044034990 msgr2=0x7fe044036400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7fe049e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe044034990 0x7fe044036400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7fe049e00640 1 -- 172.21.15.150:0/2866621850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0441728b0 con 0x7fe0440d11e0 2024-04-15T15:24:02.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c0755c0 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.456+0000 7fdf80c00640 1 -- 172.21.15.150:0/1237486247 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf7801a070 con 0x7fdf7c053c40 2024-04-15T15:24:02.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.458+0000 7f4fb5e00640 1 --2- 172.21.15.150:0/2661654753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80d4360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80098a0 tx=0x7f4fa802b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45c236a48c58913 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7f4fb4a00640 1 -- 172.21.15.150:0/2661654753 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8034070 con 0x7f4fb815ec40 2024-04-15T15:24:02.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ac005290 con 0x7fc9d015e0e0 2024-04-15T15:24:02.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7fdf80c00640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf6406c9c0 0x7fdf6406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.457+0000 7ff1cce00640 1 -- 172.21.15.150:0/3628230741 wait complete. 2024-04-15T15:24:02.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7ff1cce00640 1 Processor -- start 2024-04-15T15:24:02.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.460+0000 7fc9cd600640 1 -- 172.21.15.150:0/2606201826 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9bc01a070 con 0x7fc9d015e0e0 2024-04-15T15:24:02.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7f4fb4a00640 1 -- 172.21.15.150:0/2661654753 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa8030860 con 0x7f4fb815ec40 2024-04-15T15:24:02.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.459+0000 7f4fb4a00640 1 -- 172.21.15.150:0/2661654753 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8030c70 con 0x7f4fb815ec40 2024-04-15T15:24:02.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.460+0000 7ff1cce00640 1 -- start start 2024-04-15T15:24:02.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.460+0000 7fdf80c00640 1 -- 172.21.15.150:0/1237486247 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdf7808aed0 con 0x7fdf7c053c40 2024-04-15T15:24:02.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.460+0000 7fe049e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0440d11e0 0x7fe044172060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe04c050fc0 tx=0x7fe04c069540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7fdf83e00640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf6406c9c0 0x7fdf6406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7f4fbca00640 1 -- 172.21.15.150:0/2661654753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 msgr2=0x7f4fb80d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7fc9cd600640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc9b806e3d0 0x7fc9b8070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7f4fbca00640 1 --2- 172.21.15.150:0/2661654753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80d4360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80098a0 tx=0x7f4fa802b9a0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.460+0000 7f09abe00640 1 -- 172.21.15.150:0/1569800924 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09a001a070 con 0x7f09b40b6d30 2024-04-15T15:24:02.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7f09abe00640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f099406c8f0 0x7f099406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7fc9cd600640 1 -- 172.21.15.150:0/2606201826 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc9bc08bd60 con 0x7fc9d015e0e0 2024-04-15T15:24:02.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7f4fbca00640 1 -- 172.21.15.150:0/2661654753 shutdown_connections 2024-04-15T15:24:02.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7f4fbca00640 1 --2- 172.21.15.150:0/2661654753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80d4360 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7f4fbca00640 1 -- 172.21.15.150:0/2661654753 >> 172.21.15.150:0/2661654753 conn(0x7f4fb8073690 msgr2=0x7f4fb806cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.461+0000 7f09abe00640 1 -- 172.21.15.150:0/1569800924 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f09a0056d50 con 0x7f09b40b6d30 2024-04-15T15:24:02.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7f4fbca00640 1 -- 172.21.15.150:0/2661654753 shutdown_connections 2024-04-15T15:24:02.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7fc9d5c00640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc9b806e3d0 0x7fc9b8070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7f4fbca00640 1 -- 172.21.15.150:0/2661654753 wait complete. 2024-04-15T15:24:02.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7f09b3e00640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f099406c8f0 0x7f099406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7ff1cce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1c80701f0 0x7ff1c8157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7ff1cce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1c8072100 0x7ff1c815b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7fdf83e00640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf6406c9c0 0x7fdf6406ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdf84065750 tx=0x7fdf84063490 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7ff1cce00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 0x7ff1c81572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.462+0000 7ff1cce00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8072570 con 0x7ff1c80701f0 2024-04-15T15:24:02.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.463+0000 7f5b79600640 1 -- 172.21.15.150:0/1179096554 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b7002b070 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.463+0000 7ff1cce00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c806f770 con 0x7ff1c80b3760 2024-04-15T15:24:02.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.463+0000 7ff1c5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1c80701f0 0x7ff1c8157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.463+0000 7ff1c6800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 0x7ff1c81572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.463+0000 7ff1c5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1c80701f0 0x7ff1c8157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44636/0 (socket says 172.21.15.150:44636) 2024-04-15T15:24:02.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.463+0000 7ff1c5e00640 1 -- 172.21.15.150:0/2209348073 learned_addr learned my addr 172.21.15.150:0/2209348073 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.463+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c80726f0 con 0x7ff1c8072100 2024-04-15T15:24:02.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.463+0000 7fc9d5c00640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc9b806e3d0 0x7fc9b8070890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0001460 tx=0x7fc9c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7f09b3e00640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f099406c8f0 0x7f099406edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09a4007fd0 tx=0x7f09a4007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7f5b79600640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b5c06c9c0 0x7f5b5c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7f5b79600640 1 -- 172.21.15.150:0/1179096554 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5b7009d2f0 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7ff1c5e00640 1 -- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1c8072100 msgr2=0x7ff1c815b970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-15T15:24:02.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7ff1c5e00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1c8072100 0x7ff1c815b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7f4fbca00640 1 Processor -- start 2024-04-15T15:24:02.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7ff1c5e00640 1 -- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 msgr2=0x7ff1c81572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7f5b81400640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b5c06c9c0 0x7f5b5c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7ff1c5e00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 0x7ff1c81572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7ff1c5e00640 1 -- 172.21.15.150:0/2209348073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c815beb0 con 0x7ff1c80701f0 2024-04-15T15:24:02.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.464+0000 7f4fbca00640 1 -- start start 2024-04-15T15:24:02.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.465+0000 7fccac800640 1 Processor -- start 2024-04-15T15:24:02.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.465+0000 7ff1c5e00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1c80701f0 0x7ff1c8157c30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c750 tx=0x7ff1b800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.465+0000 7fccac800640 1 -- start start 2024-04-15T15:24:02.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.465+0000 7f4fbca00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fb80d1300 0x7f4fb80ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.465+0000 7f4fbca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb80d2c30 0x7f4fb8155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fbca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fbca00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb80d17b0 con 0x7f4fb80d2c30 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fbca00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb80d3e40 con 0x7f4fb80d1300 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fbca00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb80d1930 con 0x7f4fb815ec40 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb80d2c30 0x7f4fb8155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb80d2c30 0x7f4fb8155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44642/0 (socket says 172.21.15.150:44642) 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5400640 1 -- 172.21.15.150:0/3230193809 learned_addr learned my addr 172.21.15.150:0/3230193809 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f5b81400640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b5c06c9c0 0x7f5b5c06ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b740012e0 tx=0x7f5b74007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5e00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fb80d1300 0x7f4fb80ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5400640 1 -- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 msgr2=0x7f4fb80b5de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-15T15:24:02.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5400640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80b5de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5400640 1 -- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fb80d1300 msgr2=0x7f4fb80ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5400640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fb80d1300 0x7f4fb80ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5400640 1 -- 172.21.15.150:0/3230193809 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb80b7080 con 0x7f4fb80d2c30 2024-04-15T15:24:02.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.466+0000 7f4fb5e00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fb80d1300 0x7f4fb80ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:02.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccac800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d4360 0x7fcca40d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccac800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcca40d1300 0x7fcca40d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccac800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcca40d32c0 con 0x7fcca415ec40 2024-04-15T15:24:02.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccac800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fcca40d35a0 con 0x7fcca40d1300 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccac800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcca40d3880 con 0x7fcca40d4360 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccaa800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcca40d1300 0x7fcca40d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccaa800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcca40d1300 0x7fcca40d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44644/0 (socket says 172.21.15.150:44644) 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccaa800640 1 -- 172.21.15.150:0/193817287 learned_addr learned my addr 172.21.15.150:0/193817287 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fcca9400640 1 --2- 172.21.15.150:0/193817287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d4360 0x7fcca40d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccaa800640 1 -- 172.21.15.150:0/193817287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d4360 msgr2=0x7fcca40d2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccaa800640 1 --2- 172.21.15.150:0/193817287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d4360 0x7fcca40d2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccaa800640 1 -- 172.21.15.150:0/193817287 >> v1:172.21.15.150:6789/0 conn(0x7fcca415ec40 legacy=0x7fcca40ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fccaa800640 1 -- 172.21.15.150:0/193817287 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca40d3b60 con 0x7fcca40d1300 2024-04-15T15:24:02.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7fdf80c00640 1 -- 172.21.15.150:0/1237486247 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf78058d20 con 0x7fdf7c053c40 2024-04-15T15:24:02.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.467+0000 7f4fb5400640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb80d2c30 0x7f4fb8155de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000c430 tx=0x7f4fa000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.468+0000 7fccaa800640 1 --2- 172.21.15.150:0/193817287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d1300 0x7fcca40d1790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009e50 tx=0x7fcca002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa8e2402098c525 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7f4fae000640 1 -- 172.21.15.150:0/3230193809 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0017070 con 0x7f4fb80d2c30 2024-04-15T15:24:02.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7f4fae000640 1 -- 172.21.15.150:0/3230193809 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa00035f0 con 0x7f4fb80d2c30 2024-04-15T15:24:02.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7f4fae000640 1 -- 172.21.15.150:0/3230193809 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0003a20 con 0x7f4fb80d2c30 2024-04-15T15:24:02.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb80b6da0 con 0x7f4fb80d2c30 2024-04-15T15:24:02.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7fc9cd600640 1 -- 172.21.15.150:0/2606201826 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9bc059ab0 con 0x7fc9d015e0e0 2024-04-15T15:24:02.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7fcca8a00640 1 -- 172.21.15.150:0/193817287 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca002cb00 con 0x7fcca40d1300 2024-04-15T15:24:02.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7fcca8a00640 1 -- 172.21.15.150:0/193817287 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca002cca0 con 0x7fcca40d1300 2024-04-15T15:24:02.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7fcca8a00640 1 -- 172.21.15.150:0/193817287 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0034610 con 0x7fcca40d1300 2024-04-15T15:24:02.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7ff1be000640 1 -- 172.21.15.150:0/2209348073 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8018070 con 0x7ff1c80701f0 2024-04-15T15:24:02.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.476+0000 7ff1be000640 1 -- 172.21.15.150:0/2209348073 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b80072d0 con 0x7ff1c80701f0 2024-04-15T15:24:02.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.476+0000 7ff1be000640 1 -- 172.21.15.150:0/2209348073 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80047e0 con 0x7ff1c80701f0 2024-04-15T15:24:02.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.475+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb80cc890 con 0x7f4fb80d2c30 2024-04-15T15:24:02.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.476+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c8158280 con 0x7ff1c80701f0 2024-04-15T15:24:02.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.476+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c8158840 con 0x7ff1c80701f0 2024-04-15T15:24:02.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.477+0000 7fccac800640 1 -- 172.21.15.150:0/193817287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d1300 msgr2=0x7fcca40d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.477+0000 7fccac800640 1 --2- 172.21.15.150:0/193817287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d1300 0x7fcca40d1790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009e50 tx=0x7fcca002bf60 comp rx=0 tx=0).stop 2024-04-15T15:24:02.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.477+0000 7fccac800640 1 -- 172.21.15.150:0/193817287 shutdown_connections 2024-04-15T15:24:02.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.477+0000 7fccac800640 1 --2- 172.21.15.150:0/193817287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d4360 0x7fcca40d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.477+0000 7fccac800640 1 --2- 172.21.15.150:0/193817287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d1300 0x7fcca40d1790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.477+0000 7fccac800640 1 -- 172.21.15.150:0/193817287 >> 172.21.15.150:0/193817287 conn(0x7fcca4073690 msgr2=0x7fcca406cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.477+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c8156bd0 con 0x7ff1c80701f0 2024-04-15T15:24:02.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.478+0000 7fccac800640 1 -- 172.21.15.150:0/193817287 shutdown_connections 2024-04-15T15:24:02.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.478+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb8068c80 con 0x7f4fb80d2c30 2024-04-15T15:24:02.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.478+0000 7fccac800640 1 -- 172.21.15.150:0/193817287 wait complete. 2024-04-15T15:24:02.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.479+0000 7fccac800640 1 Processor -- start 2024-04-15T15:24:02.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.480+0000 7fccac800640 1 -- start start 2024-04-15T15:24:02.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.480+0000 7f4fae000640 1 -- 172.21.15.150:0/3230193809 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fa001a070 con 0x7f4fb80d2c30 2024-04-15T15:24:02.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.480+0000 7ff1be000640 1 -- 172.21.15.150:0/2209348073 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1b801b070 con 0x7ff1c80701f0 2024-04-15T15:24:02.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.480+0000 7f5b79600640 1 -- 172.21.15.150:0/1179096554 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b7006b790 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.481+0000 7fccac800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d1300 0x7fcca414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.481+0000 7fccac800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d4360 0x7fcca4149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.481+0000 7fe042000640 1 -- 172.21.15.150:0/2866621850 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c075490 con 0x7fe0440d11e0 2024-04-15T15:24:02.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7ff1be000640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff1a806c9c0 0x7ff1a806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe044172b90 con 0x7fe0440d11e0 2024-04-15T15:24:02.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe044173150 con 0x7fe0440d11e0 2024-04-15T15:24:02.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fccac800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca415ec40 0x7fcca41ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fccac800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40b7a70 con 0x7fcca40d4360 2024-04-15T15:24:02.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fccac800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40b78f0 con 0x7fcca415ec40 2024-04-15T15:24:02.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fccac800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcca40b7bf0 con 0x7fcca40d1300 2024-04-15T15:24:02.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7ff1be000640 1 -- 172.21.15.150:0/2209348073 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff1b808b540 con 0x7ff1c80701f0 2024-04-15T15:24:02.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fcca9400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d4360 0x7fcca4149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fcca9e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d1300 0x7fcca414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fcca9400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d4360 0x7fcca4149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44650/0 (socket says 172.21.15.150:44650) 2024-04-15T15:24:02.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fcca9400640 1 -- 172.21.15.150:0/1648694147 learned_addr learned my addr 172.21.15.150:0/1648694147 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:02.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.481+0000 7f4fae000640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f9406c9c0 0x7f4f9406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7fccaa800640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca415ec40 0x7fcca41ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7f4fae000640 1 -- 172.21.15.150:0/3230193809 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4fa008adf0 con 0x7f4fb80d2c30 2024-04-15T15:24:02.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.483+0000 7fcca9400640 1 -- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d1300 msgr2=0x7fcca414c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.483+0000 7fcca9400640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d1300 0x7fcca414c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.483+0000 7fcca9400640 1 -- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca415ec40 msgr2=0x7fcca41ec1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.483+0000 7fcca9400640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca415ec40 0x7fcca41ec1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.482+0000 7ff1c5400640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff1a806c9c0 0x7ff1a806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.483+0000 7fcca9400640 1 -- 172.21.15.150:0/1648694147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca414a9b0 con 0x7fcca40d4360 2024-04-15T15:24:02.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.483+0000 7f4fb5e00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f9406c9c0 0x7f4f9406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.483+0000 7fcca9400640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d4360 0x7fcca4149ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d8f0 tx=0x7fcc9400ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.483+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe044034db0 con 0x7fe0440d11e0 2024-04-15T15:24:02.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.485+0000 7ff1c5400640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff1a806c9c0 0x7ff1a806ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4034000 tx=0x7ff1b4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.485+0000 7f4fb5e00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f9406c9c0 0x7f4f9406ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8036000 tx=0x7f4fa803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.486+0000 7ff1be000640 1 -- 172.21.15.150:0/2209348073 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1b8022020 con 0x7ff1c80701f0 2024-04-15T15:24:02.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.490+0000 7fcc9e000640 1 -- 172.21.15.150:0/1648694147 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400b6c0 con 0x7fcca40d4360 2024-04-15T15:24:02.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.491+0000 7f4fae000640 1 -- 172.21.15.150:0/3230193809 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4fb80b7080 con 0x7f4fb80d2c30 2024-04-15T15:24:02.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.491+0000 7fcc9e000640 1 -- 172.21.15.150:0/1648694147 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc94019070 con 0x7fcca40d4360 2024-04-15T15:24:02.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.491+0000 7fcc9e000640 1 -- 172.21.15.150:0/1648694147 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400ba80 con 0x7fcca40d4360 2024-04-15T15:24:02.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.491+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca414ac90 con 0x7fcca40d4360 2024-04-15T15:24:02.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.491+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca414b250 con 0x7fcca40d4360 2024-04-15T15:24:02.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.492+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc80005290 con 0x7fcca40d4360 2024-04-15T15:24:02.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.494+0000 7fcc9e000640 1 -- 172.21.15.150:0/1648694147 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc9400bc20 con 0x7fcca40d4360 2024-04-15T15:24:02.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.495+0000 7fe042000640 1 -- 172.21.15.150:0/2866621850 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04c068eb0 con 0x7fe0440d11e0 2024-04-15T15:24:02.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.495+0000 7fe042000640 1 -- 172.21.15.150:0/2866621850 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c075830 con 0x7fe0440d11e0 2024-04-15T15:24:02.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.496+0000 7f09abe00640 1 -- 172.21.15.150:0/1569800924 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09a00593b0 con 0x7f09b40b6d30 2024-04-15T15:24:02.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.496+0000 7fcc9e000640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc8806c940 0x7fcc8806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.496+0000 7fcc9e000640 1 -- 172.21.15.150:0/1648694147 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcc9408bf30 con 0x7fcca40d4360 2024-04-15T15:24:02.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.497+0000 7fcca9e00640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc8806c940 0x7fcc8806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.498+0000 7fe042000640 1 -- 172.21.15.150:0/2866621850 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe04c09f440 con 0x7fe0440d11e0 2024-04-15T15:24:02.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.499+0000 7fcca9e00640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc8806c940 0x7fcc8806ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc98006fd0 tx=0x7fcc98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.499+0000 7fe042000640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe02c06c8f0 0x7fe02c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:02.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.500+0000 7fe042000640 1 -- 172.21.15.150:0/2866621850 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe04c107640 con 0x7fe0440d11e0 2024-04-15T15:24:02.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.501+0000 7fe049400640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe02c06c8f0 0x7fe02c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:02.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.502+0000 7fe049400640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe02c06c8f0 0x7fe02c06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe038036000 tx=0x7fe03802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:02.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.503+0000 7fcc9e000640 1 -- 172.21.15.150:0/1648694147 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc9405a3d0 con 0x7fcca40d4360 2024-04-15T15:24:02.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.509+0000 7fe042000640 1 -- 172.21.15.150:0/2866621850 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe04c0d5a30 con 0x7fe0440d11e0 2024-04-15T15:24:02.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.896+0000 7fdf76000640 1 -- 172.21.15.150:0/1237486247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdf58005580 con 0x7fdf7c053c40 2024-04-15T15:24:02.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.897+0000 7fdf80c00640 1 -- 172.21.15.150:0/1237486247 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf7805f070 con 0x7fdf7c053c40 2024-04-15T15:24:02.899 INFO:teuthology.orchestra.run.smithi150.stdout:60129542148 2024-04-15T15:24:02.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.901+0000 7fdf76000640 1 -- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf6406c9c0 msgr2=0x7fdf6406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.901+0000 7fdf76000640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf6406c9c0 0x7fdf6406ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdf84065750 tx=0x7fdf84063490 comp rx=0 tx=0).stop 2024-04-15T15:24:02.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.901+0000 7fdf76000640 1 -- 172.21.15.150:0/1237486247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf7c053c40 msgr2=0x7fdf7c0df240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.901+0000 7fdf76000640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf7c053c40 0x7fdf7c0df240 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c430 tx=0x7fdf7800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.902+0000 7fdf76000640 1 -- 172.21.15.150:0/1237486247 shutdown_connections 2024-04-15T15:24:02.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.902+0000 7fdf76000640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf7c0e7710 0x7fdf7c03b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.902+0000 7fdf76000640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf6406c9c0 0x7fdf6406ee80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.902+0000 7fdf76000640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf7c0589c0 0x7fdf7c0d1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.902+0000 7fdf76000640 1 --2- 172.21.15.150:0/1237486247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf7c053c40 0x7fdf7c0df240 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.902+0000 7fdf76000640 1 -- 172.21.15.150:0/1237486247 >> 172.21.15.150:0/1237486247 conn(0x7fdf7c058ee0 msgr2=0x7fdf7c039210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.902+0000 7fdf76000640 1 -- 172.21.15.150:0/1237486247 shutdown_connections 2024-04-15T15:24:02.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.903+0000 7fdf76000640 1 -- 172.21.15.150:0/1237486247 wait complete. 2024-04-15T15:24:02.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.923+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc9ac005580 con 0x7fc9d015e0e0 2024-04-15T15:24:02.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.924+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5b7c0b5d80 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.924+0000 7fc9cd600640 1 -- 172.21.15.150:0/2606201826 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc9bc05ddb0 con 0x7fc9d015e0e0 2024-04-15T15:24:02.926 INFO:teuthology.orchestra.run.smithi150.stdout:64424509444 2024-04-15T15:24:02.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.924+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4fb80cec40 con 0x7f4fb80d2c30 2024-04-15T15:24:02.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.925+0000 7f5b79600640 1 -- 172.21.15.150:0/1179096554 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b7006b090 con 0x7f5b7c0b6d30 2024-04-15T15:24:02.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.925+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff1c80cfda0 con 0x7ff1c80701f0 2024-04-15T15:24:02.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.925+0000 7f4fae000640 1 -- 172.21.15.150:0/3230193809 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4fa0058d00 con 0x7f4fb80d2c30 2024-04-15T15:24:02.929 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:02.929 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:02.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.926+0000 7ff1be000640 1 -- 172.21.15.150:0/2209348073 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff1c815beb0 con 0x7ff1c80701f0 2024-04-15T15:24:02.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc9b806e3d0 msgr2=0x7fc9b8070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc9b806e3d0 0x7fc9b8070890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0001460 tx=0x7fc9c0009040 comp rx=0 tx=0).stop 2024-04-15T15:24:02.930 INFO:teuthology.orchestra.run.smithi150.stdout:64424509444 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9d015e0e0 msgr2=0x7fc9d01654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9d015e0e0 0x7fc9d01654f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00e7d0 tx=0x7fc9bc00ec90 comp rx=0 tx=0).stop 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 shutdown_connections 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9d00a94a0 0x7fc9d00a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9d015e0e0 0x7fc9d01654f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc9b806e3d0 0x7fc9b8070890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 --2- 172.21.15.150:0/2606201826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9b8004160 0x7fc9d00af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.928+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 >> 172.21.15.150:0/2606201826 conn(0x7fc9d00b2460 msgr2=0x7fc9d015f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 shutdown_connections 2024-04-15T15:24:02.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b5c06c9c0 msgr2=0x7f5b5c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b5c06c9c0 0x7f5b5c06ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b740012e0 tx=0x7f5b74007d30 comp rx=0 tx=0).stop 2024-04-15T15:24:02.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b6d30 msgr2=0x7f5b7c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b6d30 0x7f5b7c152e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000e900 tx=0x7f5b7000edc0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7fc9d8600640 1 -- 172.21.15.150:0/2606201826 wait complete. 2024-04-15T15:24:02.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 shutdown_connections 2024-04-15T15:24:02.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b7c15eb50 0x7f5b7c158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b7c0b6d30 0x7f5b7c152e70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b5c06c9c0 0x7f5b5c06ee80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 --2- 172.21.15.150:0/1179096554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b7c0b3560 0x7f5b7c0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 >> 172.21.15.150:0/1179096554 conn(0x7f5b7c0d1700 msgr2=0x7f5b7c0d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 shutdown_connections 2024-04-15T15:24:02.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7f5b82800640 1 -- 172.21.15.150:0/1179096554 wait complete. 2024-04-15T15:24:02.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-04-15T15:24:02.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff1a806c9c0 msgr2=0x7ff1a806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.929+0000 7ff1cce00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff1a806c9c0 0x7ff1a806ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4034000 tx=0x7ff1b4008030 comp rx=0 tx=0).stop 2024-04-15T15:24:02.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1c80701f0 msgr2=0x7ff1c8157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1c80701f0 0x7ff1c8157c30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c750 tx=0x7ff1b800cc10 comp rx=0 tx=0).stop 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 shutdown_connections 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c80b3760 0x7ff1c81572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff1a806c9c0 0x7ff1a806ee80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1c8072100 0x7ff1c815b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 --2- 172.21.15.150:0/2209348073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1c80701f0 0x7ff1c8157c30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 >> 172.21.15.150:0/2209348073 conn(0x7ff1c80b2460 msgr2=0x7ff1c80b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 shutdown_connections 2024-04-15T15:24:02.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.930+0000 7ff1cce00640 1 -- 172.21.15.150:0/2209348073 wait complete. 2024-04-15T15:24:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.932+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f9406c9c0 msgr2=0x7f4f9406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.932+0000 7f4fbca00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f9406c9c0 0x7f4f9406ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8036000 tx=0x7f4fa803b040 comp rx=0 tx=0).stop 2024-04-15T15:24:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.932+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb80d2c30 msgr2=0x7f4fb8155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.932+0000 7f4fbca00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb80d2c30 0x7f4fb8155de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000c430 tx=0x7f4fa000c8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 shutdown_connections 2024-04-15T15:24:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7f4fbca00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f9406c9c0 0x7f4f9406ee80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7f4fbca00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fb815ec40 0x7f4fb80b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7f4fbca00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fb80d2c30 0x7f4fb8155de0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7f4fbca00640 1 --2- 172.21.15.150:0/3230193809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fb80d1300 0x7f4fb80ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 >> 172.21.15.150:0/3230193809 conn(0x7f4fb8073690 msgr2=0x7f4fb80c8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 shutdown_connections 2024-04-15T15:24:02.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7f4fbca00640 1 -- 172.21.15.150:0/3230193809 wait complete. 2024-04-15T15:24:02.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.933+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcc80005580 con 0x7fcca40d4360 2024-04-15T15:24:02.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.934+0000 7fcc9e000640 1 -- 172.21.15.150:0/1648694147 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcc94059cd0 con 0x7fcca40d4360 2024-04-15T15:24:02.938 INFO:teuthology.orchestra.run.smithi150.stdout:64424509444 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc8806c940 msgr2=0x7fcc8806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc8806c940 0x7fcc8806ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc98006fd0 tx=0x7fcc98008040 comp rx=0 tx=0).stop 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d4360 msgr2=0x7fcca4149ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d4360 0x7fcca4149ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d8f0 tx=0x7fcc9400ddb0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 shutdown_connections 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca415ec40 0x7fcca41ec1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcca40d4360 0x7fcca4149ef0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc8806c940 0x7fcc8806ee00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 --2- 172.21.15.150:0/1648694147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca40d1300 0x7fcca414c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 >> 172.21.15.150:0/1648694147 conn(0x7fcca4073690 msgr2=0x7fcca40b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 shutdown_connections 2024-04-15T15:24:02.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.937+0000 7fccac800640 1 -- 172.21.15.150:0/1648694147 wait complete. 2024-04-15T15:24:02.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.939+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe0440053c0 con 0x7fe0440d11e0 2024-04-15T15:24:02.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.940+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f09b40c8460 con 0x7f09b40b6d30 2024-04-15T15:24:02.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.940+0000 7fe042000640 1 -- 172.21.15.150:0/2866621850 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe04c0d5330 con 0x7fe0440d11e0 2024-04-15T15:24:02.945 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:02.945 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:02.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.940+0000 7f09abe00640 1 -- 172.21.15.150:0/1569800924 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f09a0058cb0 con 0x7f09b40b6d30 2024-04-15T15:24:02.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.944+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe02c06c8f0 msgr2=0x7fe02c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.944+0000 7fe050e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe02c06c8f0 0x7fe02c06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe038036000 tx=0x7fe03802e000 comp rx=0 tx=0).stop 2024-04-15T15:24:02.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.944+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0440d11e0 msgr2=0x7fe044172060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.944+0000 7fe050e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0440d11e0 0x7fe044172060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe04c050fc0 tx=0x7fe04c069540 comp rx=0 tx=0).stop 2024-04-15T15:24:02.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 shutdown_connections 2024-04-15T15:24:02.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f099406c8f0 msgr2=0x7f099406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7fe050e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe044034990 0x7fe044036400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7fe050e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe02c06c8f0 0x7fe02c06edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7fe050e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0440d4670 0x7fe044034450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f099406c8f0 0x7f099406edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09a4007fd0 tx=0x7f09a4007920 comp rx=0 tx=0).stop 2024-04-15T15:24:02.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b6d30 msgr2=0x7f09b40cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:02.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b6d30 0x7f09b40cc340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09a0001eb0 tx=0x7f09a000c960 comp rx=0 tx=0).stop 2024-04-15T15:24:02.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7fe050e00640 1 --2- 172.21.15.150:0/2866621850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0440d11e0 0x7fe044172060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 >> 172.21.15.150:0/2866621850 conn(0x7fe044009fe0 msgr2=0x7fe044039bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 shutdown_connections 2024-04-15T15:24:02.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7fe050e00640 1 -- 172.21.15.150:0/2866621850 wait complete. 2024-04-15T15:24:02.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 shutdown_connections 2024-04-15T15:24:02.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f099406c8f0 0x7f099406edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b415eb50 0x7f09b41585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b40b6d30 0x7f09b40cc340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 --2- 172.21.15.150:0/1569800924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09b40b3560 0x7f09b40ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:02.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.945+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 >> 172.21.15.150:0/1569800924 conn(0x7f09b40d1700 msgr2=0x7f09b40d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:02.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.946+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 shutdown_connections 2024-04-15T15:24:02.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:02.946+0000 7f09b9800640 1 -- 172.21.15.150:0/1569800924 wait complete. 2024-04-15T15:24:02.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-04-15T15:24:02.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.6 2024-04-15T15:24:02.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-04-15T15:24:02.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-04-15T15:24:02.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.7 2024-04-15T15:24:02.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-04-15T15:24:02.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2024-04-15T15:24:03.934 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-15T15:24:03.964 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-15T15:24:03.964 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-15T15:24:03.964 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-15T15:24:03.967 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-15T15:24:03.967 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-15T15:24:03.972 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-15T15:24:03.973 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-15T15:24:04.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.221+0000 7f24eb200640 1 Processor -- start 2024-04-15T15:24:04.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.221+0000 7f24eb200640 1 -- start start 2024-04-15T15:24:04.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.223+0000 7f24eb200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f24e4005e90 0x7f24e40062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.223+0000 7f24eb200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f24e4006810 0x7f24e4006c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.223+0000 7f24eb200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e4053720 con 0x7f24e40ead00 2024-04-15T15:24:04.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.223+0000 7f24eb200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f24e4054330 con 0x7f24e4005e90 2024-04-15T15:24:04.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.223+0000 7f24eb200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f24e4053a00 con 0x7f24e4006810 2024-04-15T15:24:04.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.224+0000 7f24e9400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f24e4005e90 0x7f24e40062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.224+0000 7f24e9400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f24e4005e90 0x7f24e40062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44666/0 (socket says 172.21.15.150:44666) 2024-04-15T15:24:04.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.224+0000 7f24e9400640 1 -- 172.21.15.150:0/989562177 learned_addr learned my addr 172.21.15.150:0/989562177 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.224+0000 7f24ea800640 1 --2- 172.21.15.150:0/989562177 >> v2:172.21.15.187:3300/0 conn(0x7f24e4006810 0x7f24e4006c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.225+0000 7f24e9400640 1 -- 172.21.15.150:0/989562177 >> v2:172.21.15.187:3300/0 conn(0x7f24e4006810 msgr2=0x7f24e4006c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.225+0000 7f24e9400640 1 --2- 172.21.15.150:0/989562177 >> v2:172.21.15.187:3300/0 conn(0x7f24e4006810 0x7f24e4006c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.225+0000 7f24e9400640 1 -- 172.21.15.150:0/989562177 >> v1:172.21.15.187:6789/0 conn(0x7f24e40ead00 legacy=0x7f24e404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.225+0000 7f24e9400640 1 -- 172.21.15.150:0/989562177 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e4053ce0 con 0x7f24e4005e90 2024-04-15T15:24:04.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.227+0000 7f24e9400640 1 --2- 172.21.15.150:0/989562177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4005e90 0x7f24e40062d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24e40528a0 tx=0x7f24d802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf271f2046413dc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.228+0000 7f24e8a00640 1 -- 172.21.15.150:0/989562177 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8035020 con 0x7f24e4005e90 2024-04-15T15:24:04.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.228+0000 7f24e8a00640 1 -- 172.21.15.150:0/989562177 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d8031890 con 0x7f24e4005e90 2024-04-15T15:24:04.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.228+0000 7f24e8a00640 1 -- 172.21.15.150:0/989562177 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8035020 con 0x7f24e4005e90 2024-04-15T15:24:04.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.230+0000 7f24eb200640 1 -- 172.21.15.150:0/989562177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4005e90 msgr2=0x7f24e40062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.230+0000 7f24eb200640 1 --2- 172.21.15.150:0/989562177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4005e90 0x7f24e40062d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24e40528a0 tx=0x7f24d802b930 comp rx=0 tx=0).stop 2024-04-15T15:24:04.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.231+0000 7f24eb200640 1 -- 172.21.15.150:0/989562177 shutdown_connections 2024-04-15T15:24:04.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.231+0000 7f24eb200640 1 --2- 172.21.15.150:0/989562177 >> v2:172.21.15.187:3300/0 conn(0x7f24e4006810 0x7f24e4006c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.231+0000 7f24eb200640 1 --2- 172.21.15.150:0/989562177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4005e90 0x7f24e40062d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.231+0000 7f24eb200640 1 -- 172.21.15.150:0/989562177 >> 172.21.15.150:0/989562177 conn(0x7f24e4009fb0 msgr2=0x7f24e400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.231+0000 7f24eb200640 1 -- 172.21.15.150:0/989562177 shutdown_connections 2024-04-15T15:24:04.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.232+0000 7f24eb200640 1 -- 172.21.15.150:0/989562177 wait complete. 2024-04-15T15:24:04.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.234+0000 7f24eb200640 1 Processor -- start 2024-04-15T15:24:04.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.235+0000 7f24eb200640 1 -- start start 2024-04-15T15:24:04.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.237+0000 7f24eb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24e4005e90 0x7f24e4033b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.237+0000 7f24eb200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4006810 0x7f24e40365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.237+0000 7f24eb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24e40ead00 0x7f24e4036b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.238+0000 7f24eb200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24e40547c0 con 0x7f24e4006810 2024-04-15T15:24:04.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.238+0000 7f24eb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24e40501a0 con 0x7f24e4005e90 2024-04-15T15:24:04.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.238+0000 7f24eb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f24e4054940 con 0x7f24e40ead00 2024-04-15T15:24:04.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.238+0000 7f24ea800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24e40ead00 0x7f24e4036b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.238+0000 7f24ea800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24e40ead00 0x7f24e4036b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:41588/0 (socket says 172.21.15.150:41588) 2024-04-15T15:24:04.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.238+0000 7f24ea800640 1 -- 172.21.15.150:0/633220447 learned_addr learned my addr 172.21.15.150:0/633220447 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.239+0000 7f24e9e00640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24e4005e90 0x7f24e4033b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.239+0000 7f24ea800640 1 -- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24e4005e90 msgr2=0x7f24e4033b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.239+0000 7f24ea800640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24e4005e90 0x7f24e4033b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.239+0000 7f24ea800640 1 -- 172.21.15.150:0/633220447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4006810 msgr2=0x7f24e40365f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-15T15:24:04.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.240+0000 7f24e9400640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4006810 0x7f24e40365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.240+0000 7f24ea800640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4006810 0x7f24e40365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.240+0000 7f24ea800640 1 -- 172.21.15.150:0/633220447 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e40db770 con 0x7f24e40ead00 2024-04-15T15:24:04.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.240+0000 7f24e9e00640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24e4005e90 0x7f24e4033b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:04.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.241+0000 7f24ea800640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24e40ead00 0x7f24e4036b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f24ec06a920 tx=0x7f24ec069540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.241+0000 7f24e9400640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4006810 0x7f24e40365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:04.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.241+0000 7f24e2000640 1 -- 172.21.15.150:0/633220447 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec098070 con 0x7f24e40ead00 2024-04-15T15:24:04.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.242+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e40dba50 con 0x7f24e40ead00 2024-04-15T15:24:04.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.242+0000 7f24e2000640 1 -- 172.21.15.150:0/633220447 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ec0754f0 con 0x7f24e40ead00 2024-04-15T15:24:04.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.243+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e40dd810 con 0x7f24e40ead00 2024-04-15T15:24:04.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.243+0000 7f24e2000640 1 -- 172.21.15.150:0/633220447 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec098070 con 0x7f24e40ead00 2024-04-15T15:24:04.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.245+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e40dbf70 con 0x7f24e40ead00 2024-04-15T15:24:04.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.246+0000 7f24e2000640 1 -- 172.21.15.150:0/633220447 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24ec075720 con 0x7f24e40ead00 2024-04-15T15:24:04.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.248+0000 7f24e2000640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f24cc06cb20 0x7f24cc06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.248+0000 7f24e2000640 1 -- 172.21.15.150:0/633220447 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f24ec1067c0 con 0x7f24e40ead00 2024-04-15T15:24:04.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.249+0000 7f24e9e00640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f24cc06cb20 0x7f24cc06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.250+0000 7f24e9e00640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f24cc06cb20 0x7f24cc06efe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00a830 tx=0x7f24dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.266+0000 7f24e2000640 1 -- 172.21.15.150:0/633220447 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24e40dbf70 con 0x7f24e40ead00 2024-04-15T15:24:04.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.270+0000 7f1eff800640 1 Processor -- start 2024-04-15T15:24:04.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.270+0000 7f1eff800640 1 -- start start 2024-04-15T15:24:04.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.271+0000 7f7b10a00640 1 Processor -- start 2024-04-15T15:24:04.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.271+0000 7f7b10a00640 1 -- start start 2024-04-15T15:24:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f1eff800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef80d4fb0 0x7f1ef8163090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f1eff800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef80d2d20 con 0x7f1ef8161fc0 2024-04-15T15:24:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f1eff800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef80cff70 con 0x7f1ef80cf9b0 2024-04-15T15:24:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f7b10a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c15ec40 0x7f7b0c0d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f7b10a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7b0c0d1300 0x7f7b0c0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f7b10a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b0c0d32c0 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f7b10a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0c0d3090 con 0x7f7b0c0d1300 2024-04-15T15:24:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f7b10a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0c0d35a0 con 0x7f7b0c15ec40 2024-04-15T15:24:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f1eff800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef80d2fc0 con 0x7f1ef80d4fb0 2024-04-15T15:24:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f1efce00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1ef8161fc0 0x7f1ef80b7af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33512/0 (socket says 172.21.15.150:33512) 2024-04-15T15:24:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.273+0000 7f1efce00640 1 -- 172.21.15.150:0/35324171 learned_addr learned my addr 172.21.15.150:0/35324171 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.274+0000 7f7b0a800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7b0c0d1300 0x7f7b0c0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.274+0000 7f1efd800640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef80d4fb0 0x7f1ef8163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.274+0000 7f7b0a800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7b0c0d1300 0x7f7b0c0d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:53998/0 (socket says 172.21.15.150:53998) 2024-04-15T15:24:04.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.274+0000 7f7b0a800640 1 -- 172.21.15.150:0/2238809400 learned_addr learned my addr 172.21.15.150:0/2238809400 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.274+0000 7f7b09e00640 1 --2- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c15ec40 0x7f7b0c0d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.274+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492317461 0 0) 0x7f1ef80d2d20 con 0x7f1ef8161fc0 2024-04-15T15:24:04.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1edc003520 con 0x7f1ef8161fc0 2024-04-15T15:24:04.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.274+0000 7f7b0a800640 1 -- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c15ec40 msgr2=0x7f7b0c0d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f7b0a800640 1 --2- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c15ec40 0x7f7b0c0d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f7b0a800640 1 -- 172.21.15.150:0/2238809400 >> v1:172.21.15.150:6789/0 conn(0x7f7b0c0d2c30 legacy=0x7f7b0c0ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1520495081 0 0) 0x7f1edc003520 con 0x7f1ef8161fc0 2024-04-15T15:24:04.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef80d4fb0 msgr2=0x7f1ef8163090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef80d4fb0 0x7f1ef8163090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 >> v1:172.21.15.187:6789/0 conn(0x7f1ef80cf9b0 legacy=0x7f1ef80d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef80d32a0 con 0x7f1ef8161fc0 2024-04-15T15:24:04.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f1ef00019f0 con 0x7f1ef8161fc0 2024-04-15T15:24:04.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.276+0000 7f7b0a800640 1 -- 172.21.15.150:0/2238809400 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c0d3880 con 0x7f7b0c0d1300 2024-04-15T15:24:04.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 >> v1:172.21.15.150:6789/0 conn(0x7f1ef8161fc0 legacy=0x7f1ef80b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.275+0000 7f1ef7400640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0039e0 0x7f1edc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.276+0000 7f1ef7400640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc004800 0x7f1edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.276+0000 7f1ef7400640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1edc005210 0x7f1edc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.276+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef80d2fc0 con 0x7f1edc005210 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.276+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc00c140 con 0x7f1edc004800 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.276+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1edc014b50 con 0x7f1edc0039e0 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.277+0000 7f1efce00640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0039e0 0x7f1edc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.277+0000 7f1ef7e00640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc004800 0x7f1edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.277+0000 7f1efd800640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1edc005210 0x7f1edc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.278+0000 7f1efce00640 1 -- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc004800 msgr2=0x7f1edc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.278+0000 7f7b09e00640 1 --2- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c15ec40 0x7f7b0c0d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.278+0000 7f1efce00640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1edc004800 0x7f1edc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.278+0000 7f1efce00640 1 -- 172.21.15.150:0/35324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1edc005210 msgr2=0x7f1edc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.278+0000 7f1efce00640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1edc005210 0x7f1edc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.278+0000 7f1efce00640 1 -- 172.21.15.150:0/35324171 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef80d32a0 con 0x7f1edc0039e0 2024-04-15T15:24:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.278+0000 7f7b0a800640 1 --2- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0c0d1300 0x7f7b0c0d17b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009870 tx=0x7f7af802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9776b84988a148a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.279+0000 7f7b08a00640 1 -- 172.21.15.150:0/2238809400 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8034070 con 0x7f7b0c0d1300 2024-04-15T15:24:04.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.279+0000 7f7b08a00640 1 -- 172.21.15.150:0/2238809400 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af8030860 con 0x7f7b0c0d1300 2024-04-15T15:24:04.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.279+0000 7f7b08a00640 1 -- 172.21.15.150:0/2238809400 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8030c70 con 0x7f7b0c0d1300 2024-04-15T15:24:04.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.279+0000 7f1efd800640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1edc005210 0x7f1edc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:04.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.280+0000 7f1efce00640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0039e0 0x7f1edc003de0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000a8f0 tx=0x7f1edc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=148af25f8a3144d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.280+0000 7f8d85800640 1 Processor -- start 2024-04-15T15:24:04.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.280+0000 7f7b10a00640 1 -- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0c0d1300 msgr2=0x7f7b0c0d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.280+0000 7f7b10a00640 1 --2- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0c0d1300 0x7f7b0c0d17b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009870 tx=0x7f7af802b9a0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.281+0000 7f7b10a00640 1 -- 172.21.15.150:0/2238809400 shutdown_connections 2024-04-15T15:24:04.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.281+0000 7f7b10a00640 1 --2- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0c0d1300 0x7f7b0c0d17b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.281+0000 7f7b10a00640 1 --2- 172.21.15.150:0/2238809400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c15ec40 0x7f7b0c0d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.280+0000 7f8d85800640 1 -- start start 2024-04-15T15:24:04.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.281+0000 7f7b10a00640 1 -- 172.21.15.150:0/2238809400 >> 172.21.15.150:0/2238809400 conn(0x7f7b0c073690 msgr2=0x7f7b0c06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.281+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000a4c0 con 0x7f1edc0039e0 2024-04-15T15:24:04.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.281+0000 7f7b10a00640 1 -- 172.21.15.150:0/2238809400 shutdown_connections 2024-04-15T15:24:04.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.281+0000 7f7b10a00640 1 -- 172.21.15.150:0/2238809400 wait complete. 2024-04-15T15:24:04.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.281+0000 7f1ef7400640 1 -- 172.21.15.150:0/35324171 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0034650 con 0x7f1edc0039e0 2024-04-15T15:24:04.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.282+0000 7f1eff800640 1 -- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0039e0 msgr2=0x7f1edc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.282+0000 7f1eff800640 1 --2- 172.21.15.150:0/35324171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1edc0039e0 0x7f1edc003de0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000a8f0 tx=0x7f1edc005830 comp rx=0 tx=0).stop 2024-04-15T15:24:04.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.282+0000 7f1efd800640 1 -- 172.21.15.150:0/35324171 reap_dead start 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.282+0000 7f8d85800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8d800b58e0 0x7f8d800afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f8d85800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d800b3900 con 0x7f8d80072810 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f8d85800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d800b3be0 con 0x7f8d80162e90 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f8d85800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f8d800b3ec0 con 0x7f8d800b58e0 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f8d7ea00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8d80162e90 0x7f8d80072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:51556/0 (socket says 172.21.15.150:51556) 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f8d7ea00640 1 -- 172.21.15.150:0/965709299 learned_addr learned my addr 172.21.15.150:0/965709299 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f8d7f400640 1 --2- 172.21.15.150:0/965709299 >> v2:172.21.15.150:3300/0 conn(0x7f8d800b58e0 0x7f8d800afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.284+0000 7f8d7d600640 1 -- 172.21.15.150:0/965709299 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3025635355 0 0) 0x7f8d800b3be0 con 0x7f8d80162e90 2024-04-15T15:24:04.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.284+0000 7f8d7d600640 1 -- 172.21.15.150:0/965709299 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d64003520 con 0x7f8d80162e90 2024-04-15T15:24:04.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.285+0000 7f8d7f400640 1 -- 172.21.15.150:0/965709299 >> v1:172.21.15.187:6789/0 conn(0x7f8d80162e90 legacy=0x7f8d80072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.285+0000 7f7b10a00640 1 Processor -- start 2024-04-15T15:24:04.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.285+0000 7f8d7d600640 1 -- 172.21.15.150:0/965709299 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675733597 0 0) 0x7f8d800b3900 con 0x7f8d80072810 2024-04-15T15:24:04.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.285+0000 7f8d7f400640 1 -- 172.21.15.150:0/965709299 >> v1:172.21.15.150:6789/0 conn(0x7f8d80072810 legacy=0x7f8d800ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.285+0000 7f8d7f400640 1 -- 172.21.15.150:0/965709299 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d800b41a0 con 0x7f8d800b58e0 2024-04-15T15:24:04.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.285+0000 7f7b10a00640 1 -- start start 2024-04-15T15:24:04.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f1eff800640 1 -- 172.21.15.150:0/35324171 shutdown_connections 2024-04-15T15:24:04.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f1eff800640 1 -- 172.21.15.150:0/35324171 >> 172.21.15.150:0/35324171 conn(0x7f1ef8073480 msgr2=0x7f1ef8073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f1eff800640 1 -- 172.21.15.150:0/35324171 shutdown_connections 2024-04-15T15:24:04.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.283+0000 7f1eff800640 1 -- 172.21.15.150:0/35324171 wait complete. 2024-04-15T15:24:04.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f8d7f400640 1 --2- 172.21.15.150:0/965709299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d800b58e0 0x7f8d800afd70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009d90 tx=0x7f8d7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc24ea7fe6d835f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f7b10a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c0d1300 0x7f7b0c0b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f7b10a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0c0d2c30 0x7f7b0c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f7b10a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0c15ec40 0x7f7b0c146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f7b10a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c0b78f0 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f7b10a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c0d3e40 con 0x7f7b0c15ec40 2024-04-15T15:24:04.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f8d7d600640 1 -- 172.21.15.150:0/965709299 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78039070 con 0x7f8d800b58e0 2024-04-15T15:24:04.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f7b10a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0c0b7a70 con 0x7f7b0c0d1300 2024-04-15T15:24:04.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f8d7d600640 1 -- 172.21.15.150:0/965709299 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7802cc80 con 0x7f8d800b58e0 2024-04-15T15:24:04.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f8d7d600640 1 -- 172.21.15.150:0/965709299 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78039070 con 0x7f8d800b58e0 2024-04-15T15:24:04.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f1eff800640 1 Processor -- start 2024-04-15T15:24:04.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f1eff800640 1 -- start start 2024-04-15T15:24:04.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.287+0000 7f7b09400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0c0d2c30 0x7f7b0c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f7b09400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0c0d2c30 0x7f7b0c153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44706/0 (socket says 172.21.15.150:44706) 2024-04-15T15:24:04.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f7b09400640 1 -- 172.21.15.150:0/1569878776 learned_addr learned my addr 172.21.15.150:0/1569878776 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f7b09400640 1 -- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c0d1300 msgr2=0x7f7b0c0b4780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-15T15:24:04.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f7b09400640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c0d1300 0x7f7b0c0b4780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f7b09400640 1 -- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0c15ec40 msgr2=0x7f7b0c146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f7b09400640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0c15ec40 0x7f7b0c146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.288+0000 7f7b09400640 1 -- 172.21.15.150:0/1569878776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c1f4440 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.289+0000 7f7b09400640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0c0d2c30 0x7f7b0c153a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b040021a0 tx=0x7f7b0400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.289+0000 7f8d85800640 1 -- 172.21.15.150:0/965709299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d800b58e0 msgr2=0x7f8d800afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.289+0000 7f8d85800640 1 --2- 172.21.15.150:0/965709299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d800b58e0 0x7f8d800afd70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009d90 tx=0x7f8d7802bed0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.289+0000 7f8d85800640 1 -- 172.21.15.150:0/965709299 shutdown_connections 2024-04-15T15:24:04.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.289+0000 7f8d85800640 1 --2- 172.21.15.150:0/965709299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d800b58e0 0x7f8d800afd70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.289+0000 7f8d85800640 1 -- 172.21.15.150:0/965709299 >> 172.21.15.150:0/965709299 conn(0x7f8d800b2460 msgr2=0x7f8d800b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.290+0000 7f8d85800640 1 -- 172.21.15.150:0/965709299 shutdown_connections 2024-04-15T15:24:04.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.290+0000 7f8d85800640 1 -- 172.21.15.150:0/965709299 wait complete. 2024-04-15T15:24:04.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.290+0000 7f1eff800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef8159e90 0x7f1ef815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.291+0000 7f1eff800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef80cba20 0x7f1ef815d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.291+0000 7f1eff800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef815dd20 0x7f1ef80ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.291+0000 7f1eff800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8163090 con 0x7f1ef80cba20 2024-04-15T15:24:04.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.291+0000 7f1eff800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81623a0 con 0x7f1ef8159e90 2024-04-15T15:24:04.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.291+0000 7f1eff800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef8163210 con 0x7f1ef815dd20 2024-04-15T15:24:04.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.291+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef80cba20 0x7f1ef815d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.291+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef80cba20 0x7f1ef815d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44712/0 (socket says 172.21.15.150:44712) 2024-04-15T15:24:04.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.291+0000 7f1ef7e00640 1 -- 172.21.15.150:0/1720009085 learned_addr learned my addr 172.21.15.150:0/1720009085 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f1efd800640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef815dd20 0x7f1ef80ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f1ef7e00640 1 -- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef815dd20 msgr2=0x7f1ef80ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f1ef7e00640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef815dd20 0x7f1ef80ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f1ef7e00640 1 -- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef8159e90 msgr2=0x7f1ef815a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f1ef7e00640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef8159e90 0x7f1ef815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f1ef7e00640 1 -- 172.21.15.150:0/1720009085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef80cbea0 con 0x7f1ef80cba20 2024-04-15T15:24:04.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.290+0000 7f7b02000640 1 -- 172.21.15.150:0/1569878776 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04015c20 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.290+0000 7f7b02000640 1 -- 172.21.15.150:0/1569878776 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b04015dc0 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.290+0000 7f7b02000640 1 -- 172.21.15.150:0/1569878776 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04003b90 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.290+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c0ccfa0 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f17f4a00640 1 Processor -- start 2024-04-15T15:24:04.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.290+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c0cd560 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.293+0000 7f1ef7e00640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef80cba20 0x7f1ef815d7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800e4b0 tx=0x7f1ee800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.293+0000 7f1ef4c00640 1 -- 172.21.15.150:0/1720009085 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80135f0 con 0x7f1ef80cba20 2024-04-15T15:24:04.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.293+0000 7f8d85800640 1 Processor -- start 2024-04-15T15:24:04.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.293+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef80c9570 con 0x7f1ef80cba20 2024-04-15T15:24:04.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.293+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef80c9b30 con 0x7f1ef80cba20 2024-04-15T15:24:04.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.293+0000 7f8d85800640 1 -- start start 2024-04-15T15:24:04.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae4005290 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.292+0000 7f17f4a00640 1 -- start start 2024-04-15T15:24:04.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.294+0000 7f17f4a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f17f00d1a10 0x7f17f00d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.294+0000 7f17f4a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17f00d32c0 con 0x7f17f00d4360 2024-04-15T15:24:04.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.294+0000 7f17f4a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17f00d35a0 con 0x7f17f015ec40 2024-04-15T15:24:04.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.294+0000 7f17f4a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f17f00d3880 con 0x7f17f00d1a10 2024-04-15T15:24:04.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.294+0000 7f29f7e00640 1 Processor -- start 2024-04-15T15:24:04.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.295+0000 7f29f7e00640 1 -- start start 2024-04-15T15:24:04.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.295+0000 7f17ede00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f17f015ec40 0x7f17f00ba390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:51570/0 (socket says 172.21.15.150:51570) 2024-04-15T15:24:04.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.295+0000 7f1ef4c00640 1 -- 172.21.15.150:0/1720009085 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee801a070 con 0x7f1ef80cba20 2024-04-15T15:24:04.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.295+0000 7f1ef4c00640 1 -- 172.21.15.150:0/1720009085 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80135f0 con 0x7f1ef80cba20 2024-04-15T15:24:04.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.295+0000 7f17ede00640 1 -- 172.21.15.150:0/378279520 learned_addr learned my addr 172.21.15.150:0/378279520 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:04.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.296+0000 7f8d85800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d80072810 0x7f8d8015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.296+0000 7f8d85800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d800b58e0 0x7f8d80160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.296+0000 7f8d85800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80162e90 0x7f8d800ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f8d85800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8d800cf2b0 con 0x7f8d80162e90 2024-04-15T15:24:04.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f8d85800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d800cf130 con 0x7f8d80072810 2024-04-15T15:24:04.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f8d85800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d800cf430 con 0x7f8d800b58e0 2024-04-15T15:24:04.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.296+0000 7f29f7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29f00e1310 0x7f29f003a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.296+0000 7f29f7e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f29f000ab40 0x7f29f0009d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f29f7e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f003a6b0 con 0x7f29f00516e0 2024-04-15T15:24:04.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f29f7e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f29f0055920 con 0x7f29f000ab40 2024-04-15T15:24:04.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f29f7e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f29f0009b50 con 0x7f29f00e1310 2024-04-15T15:24:04.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f29f6000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f29f000ab40 0x7f29f0009d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.296+0000 7f17ee800640 1 --2- 172.21.15.150:0/378279520 >> v2:172.21.15.187:3300/0 conn(0x7f17f00d1a10 0x7f17f00d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f17ee800640 1 -- 172.21.15.150:0/378279520 >> v1:172.21.15.187:6789/0 conn(0x7f17f015ec40 legacy=0x7f17f00ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f17ee800640 1 -- 172.21.15.150:0/378279520 >> v1:172.21.15.150:6789/0 conn(0x7f17f00d4360 legacy=0x7f17f00d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f17ee800640 1 -- 172.21.15.150:0/378279520 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f00d3b60 con 0x7f17f00d1a10 2024-04-15T15:24:04.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f29f6a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29f00e1310 0x7f29f003a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f29f6a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29f00e1310 0x7f29f003a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:54056/0 (socket says 172.21.15.150:54056) 2024-04-15T15:24:04.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f29f6a00640 1 -- 172.21.15.150:0/3679719978 learned_addr learned my addr 172.21.15.150:0/3679719978 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f17ede00640 1 --1- 172.21.15.150:0/378279520 >> v1:172.21.15.187:6789/0 conn(0x7f17f015ec40 0x7f17f00ba390 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-15T15:24:04.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f8d7f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80162e90 0x7f8d800ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f2ef4200640 1 Processor -- start 2024-04-15T15:24:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f8d7f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80162e90 0x7f8d800ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44722/0 (socket says 172.21.15.150:44722) 2024-04-15T15:24:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f8d7f400640 1 -- 172.21.15.150:0/1516072215 learned_addr learned my addr 172.21.15.150:0/1516072215 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f8d7f400640 1 -- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d800b58e0 msgr2=0x7f8d80160ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-15T15:24:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f8d7f400640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d800b58e0 0x7f8d80160ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f8d7f400640 1 -- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d80072810 msgr2=0x7f8d8015ce80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-15T15:24:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f8d7f400640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d80072810 0x7f8d8015ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f8d7f400640 1 -- 172.21.15.150:0/1516072215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d80161000 con 0x7f8d80162e90 2024-04-15T15:24:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.299+0000 7f17ee800640 1 --2- 172.21.15.150:0/378279520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d1a10 0x7f17f00d2c30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009d90 tx=0x7f17e802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4078d4067799c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f29f6a00640 1 -- 172.21.15.150:0/3679719978 >> v2:172.21.15.150:3300/0 conn(0x7f29f000ab40 msgr2=0x7f29f0009d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f29f6a00640 1 --2- 172.21.15.150:0/3679719978 >> v2:172.21.15.150:3300/0 conn(0x7f29f000ab40 0x7f29f0009d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f29f6a00640 1 -- 172.21.15.150:0/3679719978 >> v1:172.21.15.187:6789/0 conn(0x7f29f00516e0 legacy=0x7f29f0055db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f29f6a00640 1 -- 172.21.15.150:0/3679719978 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f0054d10 con 0x7f29f00e1310 2024-04-15T15:24:04.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.298+0000 7f2ef4200640 1 -- start start 2024-04-15T15:24:04.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.297+0000 7f1ef4c00640 1 -- 172.21.15.150:0/1720009085 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ee8013790 con 0x7f1ef80cba20 2024-04-15T15:24:04.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.299+0000 7f8d7f400640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80162e90 0x7f8d800ad5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009eb0 tx=0x7f8d78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.299+0000 7f7b02000640 1 -- 172.21.15.150:0/1569878776 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b040050d0 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.299+0000 7f25b6200640 1 Processor -- start 2024-04-15T15:24:04.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.300+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed0005290 con 0x7f1ef80cba20 2024-04-15T15:24:04.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.299+0000 7f17eca00640 1 -- 172.21.15.150:0/378279520 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8039070 con 0x7f17f00d1a10 2024-04-15T15:24:04.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.300+0000 7f17eca00640 1 -- 172.21.15.150:0/378279520 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e802cc60 con 0x7f17f00d1a10 2024-04-15T15:24:04.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.300+0000 7f17eca00640 1 -- 172.21.15.150:0/378279520 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8034660 con 0x7f17f00d1a10 2024-04-15T15:24:04.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.300+0000 7f1ef4c00640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ed406caf0 0x7f1ed406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f1ef4c00640 1 -- 172.21.15.150:0/1720009085 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1ee808b9c0 con 0x7f1ef80cba20 2024-04-15T15:24:04.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f2ef4200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2eec162e90 0x7f2eec0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f2ef4200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2eec072810 0x7f2eec0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f2ef4200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eec0b01d0 con 0x7f2eec0afd70 2024-04-15T15:24:04.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f2ef4200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f2eec0b3900 con 0x7f2eec162e90 2024-04-15T15:24:04.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f2ef4200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2eec0b3be0 con 0x7f2eec072810 2024-04-15T15:24:04.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f7b02000640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7aec06c9c0 0x7f7aec06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f7b02000640 1 -- 172.21.15.150:0/1569878776 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7b04093d90 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f17f4a00640 1 -- 172.21.15.150:0/378279520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d1a10 msgr2=0x7f17f00d2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f17f4a00640 1 --2- 172.21.15.150:0/378279520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d1a10 0x7f17f00d2c30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009d90 tx=0x7f17e802beb0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f2ef1800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2eec162e90 0x7f2eec0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f2ef1800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2eec162e90 0x7f2eec0b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44728/0 (socket says 172.21.15.150:44728) 2024-04-15T15:24:04.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f2ef1800640 1 -- 172.21.15.150:0/853561575 learned_addr learned my addr 172.21.15.150:0/853561575 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f29f6a00640 1 --2- 172.21.15.150:0/3679719978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00e1310 0x7f29f003a170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29f8074520 tx=0x7f29f80a7610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=830a1bc18d225c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.300+0000 7f25b6200640 1 -- start start 2024-04-15T15:24:04.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f17f4a00640 1 -- 172.21.15.150:0/378279520 shutdown_connections 2024-04-15T15:24:04.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.301+0000 7f17f4a00640 1 --2- 172.21.15.150:0/378279520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d1a10 0x7f17f00d2c30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f2ef1800640 1 -- 172.21.15.150:0/853561575 >> v2:172.21.15.187:3300/0 conn(0x7f2eec072810 msgr2=0x7f2eec0ce790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-15T15:24:04.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f17f4a00640 1 -- 172.21.15.150:0/378279520 >> 172.21.15.150:0/378279520 conn(0x7f17f0073690 msgr2=0x7f17f006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f1efce00640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ed406caf0 0x7f1ed406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f2ef2200640 1 --2- 172.21.15.150:0/853561575 >> v2:172.21.15.187:3300/0 conn(0x7f2eec072810 0x7f2eec0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f17f4a00640 1 -- 172.21.15.150:0/378279520 shutdown_connections 2024-04-15T15:24:04.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f25b6200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f25b015ff50 0x7f25b00d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f25b6200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b00d0f60 0x7f25b0070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f25b6200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b009baa0 con 0x7f25b009b500 2024-04-15T15:24:04.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f25b6200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f25b009c060 con 0x7f25b015ff50 2024-04-15T15:24:04.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f25b6200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25b009bd80 con 0x7f25b00d0f60 2024-04-15T15:24:04.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f7b09e00640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7aec06c9c0 0x7f7aec06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f29f5600640 1 -- 172.21.15.150:0/3679719978 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f80b3070 con 0x7f29f00e1310 2024-04-15T15:24:04.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f29f5600640 1 -- 172.21.15.150:0/3679719978 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f80ac870 con 0x7f29f00e1310 2024-04-15T15:24:04.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f29f5600640 1 -- 172.21.15.150:0/3679719978 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f80acce0 con 0x7f29f00e1310 2024-04-15T15:24:04.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.303+0000 7f25af400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f25b015ff50 0x7f25b00d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f2ef1800640 1 --2- 172.21.15.150:0/853561575 >> v2:172.21.15.187:3300/0 conn(0x7f2eec072810 0x7f2eec0ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f2ef1800640 1 -- 172.21.15.150:0/853561575 >> v1:172.21.15.187:6789/0 conn(0x7f2eec0afd70 legacy=0x7f2eec072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.302+0000 7f2ef1800640 1 -- 172.21.15.150:0/853561575 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec0b3ec0 con 0x7f2eec162e90 2024-04-15T15:24:04.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.303+0000 7f17f4a00640 1 -- 172.21.15.150:0/378279520 wait complete. 2024-04-15T15:24:04.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.303+0000 7f25af400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f25b015ff50 0x7f25b00d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44742/0 (socket says 172.21.15.150:44742) 2024-04-15T15:24:04.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.303+0000 7f25afe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b00d0f60 0x7f25b0070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.303+0000 7f25af400640 1 -- 172.21.15.150:0/1988588251 learned_addr learned my addr 172.21.15.150:0/1988588251 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.303+0000 7f29f7e00640 1 -- 172.21.15.150:0/3679719978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00e1310 msgr2=0x7f29f003a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.303+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3679719978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00e1310 0x7f29f003a170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29f8074520 tx=0x7f29f80a7610 comp rx=0 tx=0).stop 2024-04-15T15:24:04.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f1efce00640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ed406caf0 0x7f1ed406efb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0034000 tx=0x7f1ef0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f29f7e00640 1 -- 172.21.15.150:0/3679719978 shutdown_connections 2024-04-15T15:24:04.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3679719978 >> v2:172.21.15.150:3300/0 conn(0x7f29f000ab40 0x7f29f0009d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3679719978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00e1310 0x7f29f003a170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f29f7e00640 1 -- 172.21.15.150:0/3679719978 >> 172.21.15.150:0/3679719978 conn(0x7f29f0056cd0 msgr2=0x7f29f00570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f29f7e00640 1 -- 172.21.15.150:0/3679719978 shutdown_connections 2024-04-15T15:24:04.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f7b09e00640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7aec06c9c0 0x7f7aec06ee80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00a810 tx=0x7f7afc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f29f7e00640 1 -- 172.21.15.150:0/3679719978 wait complete. 2024-04-15T15:24:04.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f25af400640 1 -- 172.21.15.150:0/1988588251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b00d0f60 msgr2=0x7f25b0070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:04.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f2ef1800640 1 --2- 172.21.15.150:0/853561575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec0b58e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2eec06faa0 tx=0x7f2ee002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0d5b98afbee874 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f25af400640 1 --2- 172.21.15.150:0/1988588251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b00d0f60 0x7f25b0070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f25af400640 1 -- 172.21.15.150:0/1988588251 >> v1:172.21.15.150:6789/0 conn(0x7f25b009b500 legacy=0x7f25b00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.304+0000 7f25af400640 1 -- 172.21.15.150:0/1988588251 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00704d0 con 0x7f25b015ff50 2024-04-15T15:24:04.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.305+0000 7f17f4a00640 1 Processor -- start 2024-04-15T15:24:04.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.305+0000 7f17f4a00640 1 -- start start 2024-04-15T15:24:04.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f29f7e00640 1 Processor -- start 2024-04-15T15:24:04.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f17f4a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f00d1a10 0x7f17f01617a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f17f4a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d4360 0x7f17f00cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f29f7e00640 1 -- start start 2024-04-15T15:24:04.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f17f4a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17f015ec40 0x7f17f00cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f17f4a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01697c0 con 0x7f17f015ec40 2024-04-15T15:24:04.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f17f4a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0169640 con 0x7f17f00d4360 2024-04-15T15:24:04.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f17f4a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f17f0169940 con 0x7f17f00d1a10 2024-04-15T15:24:04.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f25af400640 1 --2- 172.21.15.150:0/1988588251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b015ff50 0x7f25b00d0290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009e50 tx=0x7f25a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb34017db9e85a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f8d76a00640 1 -- 172.21.15.150:0/1516072215 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78042020 con 0x7f8d80162e90 2024-04-15T15:24:04.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.306+0000 7f2eebe00640 1 -- 172.21.15.150:0/853561575 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0035020 con 0x7f2eec162e90 2024-04-15T15:24:04.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d800afe30 con 0x7f8d80162e90 2024-04-15T15:24:04.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ee800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17f015ec40 0x7f17f00cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ede00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f00d1a10 0x7f17f01617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ede00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f00d1a10 0x7f17f01617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:41656/0 (socket says 172.21.15.150:41656) 2024-04-15T15:24:04.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ede00640 1 -- 172.21.15.150:0/1694224196 learned_addr learned my addr 172.21.15.150:0/1694224196 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ed400640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d4360 0x7f17f00cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f7b02000640 1 -- 172.21.15.150:0/1569878776 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b040621a0 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d800b03f0 con 0x7f8d80162e90 2024-04-15T15:24:04.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f000ab40 0x7f29f00d6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00516e0 0x7f29f004c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f00e1310 0x7f29f004dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ee800640 1 -- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f00d1a10 msgr2=0x7f17f01617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f29f7e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00555b0 con 0x7f29f000ab40 2024-04-15T15:24:04.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f29f7e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00552d0 con 0x7f29f00516e0 2024-04-15T15:24:04.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f29f7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29f00548a0 con 0x7f29f00e1310 2024-04-15T15:24:04.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ee800640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f00d1a10 0x7f17f01617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ee800640 1 -- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d4360 msgr2=0x7f17f00cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ee800640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d4360 0x7f17f00cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.307+0000 7f17ee800640 1 -- 172.21.15.150:0/1694224196 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f00b69a0 con 0x7f17f015ec40 2024-04-15T15:24:04.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f000ab40 0x7f29f00d6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f000ab40 0x7f29f00d6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44760/0 (socket says 172.21.15.150:44760) 2024-04-15T15:24:04.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6a00640 1 -- 172.21.15.150:0/3262775001 learned_addr learned my addr 172.21.15.150:0/3262775001 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6000640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00516e0 0x7f29f004c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f7400640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f00e1310 0x7f29f004dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6a00640 1 -- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f00e1310 msgr2=0x7f29f004dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6a00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f00e1310 0x7f29f004dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6a00640 1 -- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00516e0 msgr2=0x7f29f004c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6a00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00516e0 0x7f29f004c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6a00640 1 -- 172.21.15.150:0/3262775001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f00d6a00 con 0x7f29f000ab40 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f29f6000640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00516e0 0x7f29f004c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f17ee800640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17f015ec40 0x7f17f00cbf40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17e8033000 tx=0x7f17e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.308+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d800afbc0 con 0x7f8d80162e90 2024-04-15T15:24:04.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.309+0000 7f29f6a00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f000ab40 0x7f29f00d6060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f29f8069110 tx=0x7f29f806b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.312+0000 7f29ef400640 1 -- 172.21.15.150:0/3262775001 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f80be070 con 0x7f29f000ab40 2024-04-15T15:24:04.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.312+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f00e1940 con 0x7f29f000ab40 2024-04-15T15:24:04.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.312+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f00e0820 con 0x7f29f000ab40 2024-04-15T15:24:04.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.312+0000 7f1ef4c00640 1 -- 172.21.15.150:0/1720009085 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ee808de20 con 0x7f1ef80cba20 2024-04-15T15:24:04.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f17e6000640 1 -- 172.21.15.150:0/1694224196 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8044020 con 0x7f17f015ec40 2024-04-15T15:24:04.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f25ae000640 1 -- 172.21.15.150:0/1988588251 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a802ca80 con 0x7f25b015ff50 2024-04-15T15:24:04.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f00b6c80 con 0x7f17f015ec40 2024-04-15T15:24:04.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f2eebe00640 1 -- 172.21.15.150:0/853561575 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee0031870 con 0x7f2eec162e90 2024-04-15T15:24:04.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f2eebe00640 1 -- 172.21.15.150:0/853561575 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0031c40 con 0x7f2eec162e90 2024-04-15T15:24:04.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f013c6e0 con 0x7f17f015ec40 2024-04-15T15:24:04.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f17e6000640 1 -- 172.21.15.150:0/1694224196 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e803a040 con 0x7f17f015ec40 2024-04-15T15:24:04.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f17e6000640 1 -- 172.21.15.150:0/1694224196 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8039070 con 0x7f17f015ec40 2024-04-15T15:24:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f25ae000640 1 -- 172.21.15.150:0/1988588251 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a802cc20 con 0x7f25b015ff50 2024-04-15T15:24:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.313+0000 7f25ae000640 1 -- 172.21.15.150:0/1988588251 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a80343f0 con 0x7f25b015ff50 2024-04-15T15:24:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.314+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f00057e0 con 0x7f29f000ab40 2024-04-15T15:24:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.314+0000 7f29ef400640 1 -- 172.21.15.150:0/3262775001 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f80ac870 con 0x7f29f000ab40 2024-04-15T15:24:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.314+0000 7f29ef400640 1 -- 172.21.15.150:0/3262775001 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f80b3040 con 0x7f29f000ab40 2024-04-15T15:24:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.314+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c8005290 con 0x7f17f015ec40 2024-04-15T15:24:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.315+0000 7f2ef4200640 1 -- 172.21.15.150:0/853561575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 msgr2=0x7f2eec0b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.315+0000 7f2ef4200640 1 --2- 172.21.15.150:0/853561575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec0b58e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2eec06faa0 tx=0x7f2ee002b910 comp rx=0 tx=0).stop 2024-04-15T15:24:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.316+0000 7f2ef4200640 1 -- 172.21.15.150:0/853561575 shutdown_connections 2024-04-15T15:24:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.316+0000 7f2ef4200640 1 --2- 172.21.15.150:0/853561575 >> v2:172.21.15.187:3300/0 conn(0x7f2eec072810 0x7f2eec0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.316+0000 7f2ef4200640 1 --2- 172.21.15.150:0/853561575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec0b58e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.312+0000 7f8d76a00640 1 -- 172.21.15.150:0/1516072215 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7803a040 con 0x7f8d80162e90 2024-04-15T15:24:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.312+0000 7f8d76a00640 1 -- 172.21.15.150:0/1516072215 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78039070 con 0x7f8d80162e90 2024-04-15T15:24:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.316+0000 7f8d76a00640 1 -- 172.21.15.150:0/1516072215 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d78045020 con 0x7f8d80162e90 2024-04-15T15:24:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f8d76a00640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d6006c8f0 0x7f8d6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f8d76a00640 1 -- 172.21.15.150:0/1516072215 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8d780ae2e0 con 0x7f8d80162e90 2024-04-15T15:24:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.318+0000 7f8d7ea00640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d6006c8f0 0x7f8d6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.316+0000 7f2ef4200640 1 -- 172.21.15.150:0/853561575 >> 172.21.15.150:0/853561575 conn(0x7f2eec0b2460 msgr2=0x7f2eec0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f2ef4200640 1 -- 172.21.15.150:0/853561575 shutdown_connections 2024-04-15T15:24:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f2ef4200640 1 -- 172.21.15.150:0/853561575 wait complete. 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f25b6200640 1 -- 172.21.15.150:0/1988588251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b015ff50 msgr2=0x7f25b00d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f25b6200640 1 --2- 172.21.15.150:0/1988588251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b015ff50 0x7f25b00d0290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009e50 tx=0x7f25a802bee0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f25b6200640 1 -- 172.21.15.150:0/1988588251 shutdown_connections 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f25b6200640 1 --2- 172.21.15.150:0/1988588251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b00d0f60 0x7f25b0070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.318+0000 7f25b6200640 1 --2- 172.21.15.150:0/1988588251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b015ff50 0x7f25b00d0290 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.318+0000 7f25b6200640 1 -- 172.21.15.150:0/1988588251 >> 172.21.15.150:0/1988588251 conn(0x7f25b00d26c0 msgr2=0x7f25b00d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.318+0000 7f25b6200640 1 -- 172.21.15.150:0/1988588251 shutdown_connections 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.318+0000 7f25b6200640 1 -- 172.21.15.150:0/1988588251 wait complete. 2024-04-15T15:24:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.317+0000 7f29ef400640 1 -- 172.21.15.150:0/3262775001 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29f80c1070 con 0x7f29f000ab40 2024-04-15T15:24:04.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.319+0000 7f2ef4200640 1 Processor -- start 2024-04-15T15:24:04.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.319+0000 7f2ef4200640 1 -- start start 2024-04-15T15:24:04.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.319+0000 7f29ef400640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d406c8f0 0x7f29d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.318+0000 7f17e6000640 1 -- 172.21.15.150:0/1694224196 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17e8047020 con 0x7f17f015ec40 2024-04-15T15:24:04.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.319+0000 7f17e6000640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f17d006c8f0 0x7f17d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.320+0000 7f8d7ea00640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d6006c8f0 0x7f8d6006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d70000f60 tx=0x7f8d700010a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.320+0000 7f17ede00640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f17d006c8f0 0x7f17d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef4200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eec072810 0x7f2eec1ed310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef4200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eec0afd70 0x7f2eec0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef4200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec1650d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.320+0000 7f29f6000640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d406c8f0 0x7f29d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f29f6000640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d406c8f0 0x7f29d406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f29f004d320 tx=0x7f29e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef4200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec0cee50 con 0x7f2eec162e90 2024-04-15T15:24:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef4200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec0cecd0 con 0x7f2eec072810 2024-04-15T15:24:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef4200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec0cefd0 con 0x7f2eec0afd70 2024-04-15T15:24:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef2200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec1650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef2200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec1650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44766/0 (socket says 172.21.15.150:44766) 2024-04-15T15:24:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.321+0000 7f2ef2200640 1 -- 172.21.15.150:0/2819002878 learned_addr learned my addr 172.21.15.150:0/2819002878 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.322+0000 7f29ef400640 1 -- 172.21.15.150:0/3262775001 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f29f80f4eb0 con 0x7f29f000ab40 2024-04-15T15:24:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.322+0000 7f2ef1800640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eec072810 0x7f2eec1ed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.322+0000 7f2ef0e00640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eec0afd70 0x7f2eec0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.322+0000 7f2ef2200640 1 -- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eec0afd70 msgr2=0x7f2eec0b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.322+0000 7f2ef2200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eec0afd70 0x7f2eec0b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.322+0000 7f2ef2200640 1 -- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eec072810 msgr2=0x7f2eec1ed310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.322+0000 7f2ef2200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eec072810 0x7f2eec1ed310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.322+0000 7f2ef2200640 1 -- 172.21.15.150:0/2819002878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec1ec270 con 0x7f2eec162e90 2024-04-15T15:24:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.323+0000 7f2ef2200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec1650d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400cd70 tx=0x7f2ee400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.323+0000 7f25b6200640 1 Processor -- start 2024-04-15T15:24:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.323+0000 7f17e6000640 1 -- 172.21.15.150:0/1694224196 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f17e807aa10 con 0x7f17f015ec40 2024-04-15T15:24:04.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.323+0000 7f17ede00640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f17d006c8f0 0x7f17d006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f17e0006fd0 tx=0x7f17e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.323+0000 7f8d76a00640 1 -- 172.21.15.150:0/1516072215 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d7804c030 con 0x7f8d80162e90 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.324+0000 7f25b6200640 1 -- start start 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.325+0000 7f25b6200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25b009b500 0x7f25b00c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.325+0000 7f25b6200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00d0f60 0x7f25b00b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.325+0000 7f25b6200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b015ff50 0x7f25b00cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.325+0000 7f25b6200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25b00ce530 con 0x7f25b00d0f60 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.325+0000 7f25b6200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0160330 con 0x7f25b009b500 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.325+0000 7f25b6200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25b0071210 con 0x7f25b015ff50 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.325+0000 7f25aea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00d0f60 0x7f25b00b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.325+0000 7f25aea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00d0f60 0x7f25b00b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44774/0 (socket says 172.21.15.150:44774) 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25aea00640 1 -- 172.21.15.150:0/3801116682 learned_addr learned my addr 172.21.15.150:0/3801116682 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25af400640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25b009b500 0x7f25b00c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25afe00640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b015ff50 0x7f25b00cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25aea00640 1 -- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b015ff50 msgr2=0x7f25b00cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25aea00640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b015ff50 0x7f25b00cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25aea00640 1 -- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25b009b500 msgr2=0x7f25b00c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25aea00640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25b009b500 0x7f25b00c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25aea00640 1 -- 172.21.15.150:0/3801116682 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00cad00 con 0x7f25b00d0f60 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.326+0000 7f25aea00640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00d0f60 0x7f25b00b5380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c430 tx=0x7f25a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.328+0000 7f17e6000640 1 -- 172.21.15.150:0/1694224196 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17e804e030 con 0x7f17f015ec40 2024-04-15T15:24:04.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.338+0000 7f29ef400640 1 -- 172.21.15.150:0/3262775001 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29f80c8020 con 0x7f29f000ab40 2024-04-15T15:24:04.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.338+0000 7f2ee9600640 1 -- 172.21.15.150:0/2819002878 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4003b90 con 0x7f2eec162e90 2024-04-15T15:24:04.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.339+0000 7f2ee9600640 1 -- 172.21.15.150:0/2819002878 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee4003d30 con 0x7f2eec162e90 2024-04-15T15:24:04.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.339+0000 7f25a7400640 1 -- 172.21.15.150:0/3801116682 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a00052d0 con 0x7f25b00d0f60 2024-04-15T15:24:04.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.339+0000 7f25a7400640 1 -- 172.21.15.150:0/3801116682 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a0005490 con 0x7f25b00d0f60 2024-04-15T15:24:04.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.339+0000 7f25a7400640 1 -- 172.21.15.150:0/3801116682 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0007260 con 0x7f25b00d0f60 2024-04-15T15:24:04.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.339+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b00cafe0 con 0x7f25b00d0f60 2024-04-15T15:24:04.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.340+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b00cb5a0 con 0x7f25b00d0f60 2024-04-15T15:24:04.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.339+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec1ec550 con 0x7f2eec162e90 2024-04-15T15:24:04.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.339+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec1ecb10 con 0x7f2eec162e90 2024-04-15T15:24:04.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.340+0000 7f2ee9600640 1 -- 172.21.15.150:0/2819002878 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4004240 con 0x7f2eec162e90 2024-04-15T15:24:04.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.341+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec8005290 con 0x7f2eec162e90 2024-04-15T15:24:04.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.345+0000 7f2ee9600640 1 -- 172.21.15.150:0/2819002878 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ee40043e0 con 0x7f2eec162e90 2024-04-15T15:24:04.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.346+0000 7f2ee9600640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ecc06c8f0 0x7f2ecc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.349+0000 7f2ee9600640 1 -- 172.21.15.150:0/2819002878 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2ee40583f0 con 0x7f2eec162e90 2024-04-15T15:24:04.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.350+0000 7f2ef1800640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ecc06c8f0 0x7f2ecc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.350+0000 7f25a7400640 1 -- 172.21.15.150:0/3801116682 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25a0007400 con 0x7f25b00d0f60 2024-04-15T15:24:04.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.351+0000 7f2ef1800640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ecc06c8f0 0x7f2ecc06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0036000 tx=0x7f2ee003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.352+0000 7f25a7400640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f259006c9c0 0x7f259006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:04.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.352+0000 7f25a7400640 1 -- 172.21.15.150:0/3801116682 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f25a008b010 con 0x7f25b00d0f60 2024-04-15T15:24:04.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.352+0000 7f25af400640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f259006c9c0 0x7f259006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:04.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.353+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b0068c80 con 0x7f25b00d0f60 2024-04-15T15:24:04.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.356+0000 7f2ee9600640 1 -- 172.21.15.150:0/2819002878 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ee405aa50 con 0x7f2eec162e90 2024-04-15T15:24:04.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.357+0000 7f25af400640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f259006c9c0 0x7f259006ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25a8035000 tx=0x7f25a8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:04.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.367+0000 7f25a7400640 1 -- 172.21.15.150:0/3801116682 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25a0058dd0 con 0x7f25b00d0f60 2024-04-15T15:24:04.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.723+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f24e4005390 con 0x7f24e40ead00 2024-04-15T15:24:04.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.724+0000 7f24e2000640 1 -- 172.21.15.150:0/633220447 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24ec090070 con 0x7f24e40ead00 2024-04-15T15:24:04.729 INFO:teuthology.orchestra.run.smithi150.stdout:60129542150 2024-04-15T15:24:04.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f24cc06cb20 msgr2=0x7f24cc06efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f24cc06cb20 0x7f24cc06efe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00a830 tx=0x7f24dc008040 comp rx=0 tx=0).stop 2024-04-15T15:24:04.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24e40ead00 msgr2=0x7f24e4036b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24e40ead00 0x7f24e4036b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f24ec06a920 tx=0x7f24ec069540 comp rx=0 tx=0).stop 2024-04-15T15:24:04.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 shutdown_connections 2024-04-15T15:24:04.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f24cc06cb20 0x7f24cc06efe0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00a830 tx=0x7f24dc008040 comp rx=0 tx=0).stop 2024-04-15T15:24:04.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24e40ead00 0x7f24e4036b30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e4006810 0x7f24e40365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 --2- 172.21.15.150:0/633220447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24e4005e90 0x7f24e4033b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.728+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 >> 172.21.15.150:0/633220447 conn(0x7f24e4009fb0 msgr2=0x7f24e4047a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.729+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 shutdown_connections 2024-04-15T15:24:04.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.730+0000 7f24eb200640 1 -- 172.21.15.150:0/633220447 wait complete. 2024-04-15T15:24:04.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.743+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1ed0005850 con 0x7f1ef80cba20 2024-04-15T15:24:04.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.744+0000 7f1ef4c00640 1 -- 172.21.15.150:0/1720009085 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ee8059ea0 con 0x7f1ef80cba20 2024-04-15T15:24:04.749 INFO:teuthology.orchestra.run.smithi150.stdout:64424509446 2024-04-15T15:24:04.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.745+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8d800b2a50 con 0x7f8d80162e90 2024-04-15T15:24:04.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.747+0000 7f8d76a00640 1 -- 172.21.15.150:0/1516072215 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8d7807c700 con 0x7f8d80162e90 2024-04-15T15:24:04.749 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:04.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.749+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ed406caf0 msgr2=0x7f1ed406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.749+0000 7f1eff800640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ed406caf0 0x7f1ed406efb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0034000 tx=0x7f1ef0009030 comp rx=0 tx=0).stop 2024-04-15T15:24:04.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.749+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef80cba20 msgr2=0x7f1ef815d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.749+0000 7f1eff800640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef80cba20 0x7f1ef815d7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800e4b0 tx=0x7f1ee800e970 comp rx=0 tx=0).stop 2024-04-15T15:24:04.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.750+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 shutdown_connections 2024-04-15T15:24:04.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.750+0000 7f1eff800640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ed406caf0 0x7f1ed406efb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.750+0000 7f1eff800640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef815dd20 0x7f1ef80ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.750+0000 7f1eff800640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef80cba20 0x7f1ef815d7e0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800e4b0 tx=0x7f1ee800e970 comp rx=0 tx=0).stop 2024-04-15T15:24:04.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.750+0000 7f1eff800640 1 --2- 172.21.15.150:0/1720009085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef8159e90 0x7f1ef815a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.750+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 >> 172.21.15.150:0/1720009085 conn(0x7f1ef8073480 msgr2=0x7f1ef80b9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.751+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 shutdown_connections 2024-04-15T15:24:04.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.751+0000 7f1eff800640 1 -- 172.21.15.150:0/1720009085 wait complete. 2024-04-15T15:24:04.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.751+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d6006c8f0 msgr2=0x7f8d6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.751+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f17c8005850 con 0x7f17f015ec40 2024-04-15T15:24:04.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.751+0000 7f8d85800640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d6006c8f0 0x7f8d6006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d70000f60 tx=0x7f8d700010a0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.751+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80162e90 msgr2=0x7f8d800ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.751+0000 7f8d85800640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80162e90 0x7f8d800ad5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009eb0 tx=0x7f8d78004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:04.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.751+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7ae4005580 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.752+0000 7f17e6000640 1 -- 172.21.15.150:0/1694224196 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f17e803ee20 con 0x7f17f015ec40 2024-04-15T15:24:04.754 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f7b02000640 1 -- 172.21.15.150:0/1569878776 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b04061aa0 con 0x7f7b0c0d2c30 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 shutdown_connections 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f8d85800640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80162e90 0x7f8d800ad5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f8d85800640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d6006c8f0 0x7f8d6006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f8d85800640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d800b58e0 0x7f8d80160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f8d85800640 1 --2- 172.21.15.150:0/1516072215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d80072810 0x7f8d8015ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 >> 172.21.15.150:0/1516072215 conn(0x7f8d800b2460 msgr2=0x7f8d800cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 shutdown_connections 2024-04-15T15:24:04.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.753+0000 7f8d85800640 1 -- 172.21.15.150:0/1516072215 wait complete. 2024-04-15T15:24:04.756 INFO:teuthology.orchestra.run.smithi150.stdout:64424509445 2024-04-15T15:24:04.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.755+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f17d006c8f0 msgr2=0x7f17d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f17d006c8f0 0x7f17d006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f17e0006fd0 tx=0x7f17e0008040 comp rx=0 tx=0).stop 2024-04-15T15:24:04.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17f015ec40 msgr2=0x7f17f00cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17f015ec40 0x7f17f00cbf40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17e8033000 tx=0x7f17e8004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:04.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.2 2024-04-15T15:24:04.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7aec06c9c0 msgr2=0x7f7aec06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7aec06c9c0 0x7f7aec06ee80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00a810 tx=0x7f7afc008040 comp rx=0 tx=0).stop 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0c0d2c30 msgr2=0x7f7b0c153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0c0d2c30 0x7f7b0c153a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b040021a0 tx=0x7f7b0400cf80 comp rx=0 tx=0).stop 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 shutdown_connections 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17f015ec40 0x7f17f00cbf40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f00d4360 0x7f17f00cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f17d006c8f0 0x7f17d006edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 --2- 172.21.15.150:0/1694224196 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f00d1a10 0x7f17f01617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 >> 172.21.15.150:0/1694224196 conn(0x7f17f0073690 msgr2=0x7f17f006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.760 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:04.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 shutdown_connections 2024-04-15T15:24:04.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0c15ec40 0x7f7b0c146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0c0d2c30 0x7f7b0c153a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7aec06c9c0 0x7f7aec06ee80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 --2- 172.21.15.150:0/1569878776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b0c0d1300 0x7f7b0c0b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 >> 172.21.15.150:0/1569878776 conn(0x7f7b0c073690 msgr2=0x7f7b0c0b9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.756+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 shutdown_connections 2024-04-15T15:24:04.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.757+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 shutdown_connections 2024-04-15T15:24:04.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.757+0000 7f7b10a00640 1 -- 172.21.15.150:0/1569878776 wait complete. 2024-04-15T15:24:04.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.757+0000 7f17f4a00640 1 -- 172.21.15.150:0/1694224196 wait complete. 2024-04-15T15:24:04.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.771+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f29f00e1e60 con 0x7f29f000ab40 2024-04-15T15:24:04.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.772+0000 7f29ef400640 1 -- 172.21.15.150:0/3262775001 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29f00e1e60 con 0x7f29f000ab40 2024-04-15T15:24:04.773 INFO:teuthology.orchestra.run.smithi150.stdout:64424509446 2024-04-15T15:24:04.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d406c8f0 msgr2=0x7f29d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d406c8f0 0x7f29d406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f29f004d320 tx=0x7f29e8008040 comp rx=0 tx=0).stop 2024-04-15T15:24:04.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f000ab40 msgr2=0x7f29f00d6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f000ab40 0x7f29f00d6060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f29f8069110 tx=0x7f29f806b5b0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 shutdown_connections 2024-04-15T15:24:04.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d406c8f0 0x7f29d406edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f00e1310 0x7f29f004dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f00516e0 0x7f29f004c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 --2- 172.21.15.150:0/3262775001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f000ab40 0x7f29f00d6060 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.775+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 >> 172.21.15.150:0/3262775001 conn(0x7f29f0056cd0 msgr2=0x7f29f003c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.776+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 shutdown_connections 2024-04-15T15:24:04.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.776+0000 7f29f7e00640 1 -- 172.21.15.150:0/3262775001 wait complete. 2024-04-15T15:24:04.778 INFO:teuthology.orchestra.run.smithi150.stdout:64424509446 2024-04-15T15:24:04.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.776+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2ec8005850 con 0x7f2eec162e90 2024-04-15T15:24:04.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.777+0000 7f2ee9600640 1 -- 172.21.15.150:0/2819002878 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ee405a350 con 0x7f2eec162e90 2024-04-15T15:24:04.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-04-15T15:24:04.783 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:04.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.780+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ecc06c8f0 msgr2=0x7f2ecc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.780+0000 7f2ef4200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ecc06c8f0 0x7f2ecc06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0036000 tx=0x7f2ee003c040 comp rx=0 tx=0).stop 2024-04-15T15:24:04.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.780+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 msgr2=0x7f2eec1650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.780+0000 7f2ef4200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec1650d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400cd70 tx=0x7f2ee400a660 comp rx=0 tx=0).stop 2024-04-15T15:24:04.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-04-15T15:24:04.784 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:04.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.781+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 shutdown_connections 2024-04-15T15:24:04.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.781+0000 7f2ef4200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2eec162e90 0x7f2eec1650d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.781+0000 7f2ef4200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ecc06c8f0 0x7f2ecc06edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.781+0000 7f2ef4200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2eec0afd70 0x7f2eec0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.781+0000 7f2ef4200640 1 --2- 172.21.15.150:0/2819002878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eec072810 0x7f2eec1ed310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.781+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 >> 172.21.15.150:0/2819002878 conn(0x7f2eec0b2460 msgr2=0x7f2eec0b5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.781+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 shutdown_connections 2024-04-15T15:24:04.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.781+0000 7f2ef4200640 1 -- 172.21.15.150:0/2819002878 wait complete. 2024-04-15T15:24:04.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.7 2024-04-15T15:24:04.789 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:04.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.6 2024-04-15T15:24:04.789 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:04.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.792+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f25b00cee50 con 0x7f25b00d0f60 2024-04-15T15:24:04.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.793+0000 7f25a7400640 1 -- 172.21.15.150:0/3801116682 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f25a005d0d0 con 0x7f25b00d0f60 2024-04-15T15:24:04.796 INFO:teuthology.orchestra.run.smithi150.stdout:64424509446 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.795+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f259006c9c0 msgr2=0x7f259006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.795+0000 7f25b6200640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f259006c9c0 0x7f259006ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25a8035000 tx=0x7f25a8030000 comp rx=0 tx=0).stop 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.795+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00d0f60 msgr2=0x7f25b00b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.795+0000 7f25b6200640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00d0f60 0x7f25b00b5380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c430 tx=0x7f25a000c8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.796+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 shutdown_connections 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.796+0000 7f25b6200640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f259006c9c0 0x7f259006ee80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.796+0000 7f25b6200640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25b015ff50 0x7f25b00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.796+0000 7f25b6200640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00d0f60 0x7f25b00b5380 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.796+0000 7f25b6200640 1 --2- 172.21.15.150:0/3801116682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25b009b500 0x7f25b00c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.796+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 >> 172.21.15.150:0/3801116682 conn(0x7f25b00d26c0 msgr2=0x7f25b00b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:04.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.796+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 shutdown_connections 2024-04-15T15:24:04.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:04.796+0000 7f25b6200640 1 -- 172.21.15.150:0/3801116682 wait complete. 2024-04-15T15:24:04.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2024-04-15T15:24:04.801 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:04.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-04-15T15:24:04.804 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:04.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-04-15T15:24:04.814 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:04.814 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-15T15:24:04.815 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:24:05.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.053+0000 7f3d0bc00640 1 Processor -- start 2024-04-15T15:24:05.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.054+0000 7f3d0bc00640 1 -- start start 2024-04-15T15:24:05.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.055+0000 7f3d0bc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d041461f0 0x7f3d040ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.055+0000 7f3d0bc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040cedf0 0x7f3d040b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.055+0000 7f3d0bc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3d040b3e10 0x7f3d040cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.055+0000 7f3d0bc00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f3d040a9640 con 0x7f3d041461f0 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.055+0000 7f3d0bc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3d040cf250 con 0x7f3d040b3e10 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.055+0000 7f3d0bc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d040b4be0 con 0x7f3d040cedf0 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.055+0000 7f3d09200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d041461f0 0x7f3d040ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d09200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d041461f0 0x7f3d040ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44782/0 (socket says 172.21.15.150:44782) 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d09200640 1 -- 172.21.15.150:0/1412830028 learned_addr learned my addr 172.21.15.150:0/1412830028 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d03e00640 1 --2- 172.21.15.150:0/1412830028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040cedf0 0x7f3d040b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d09c00640 1 --2- 172.21.15.150:0/1412830028 >> v2:172.21.15.187:3300/0 conn(0x7f3d040b3e10 0x7f3d040cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d09200640 1 -- 172.21.15.150:0/1412830028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040cedf0 msgr2=0x7f3d040b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d09200640 1 --2- 172.21.15.150:0/1412830028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040cedf0 0x7f3d040b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d09200640 1 -- 172.21.15.150:0/1412830028 >> v2:172.21.15.187:3300/0 conn(0x7f3d040b3e10 msgr2=0x7f3d040cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d09200640 1 --2- 172.21.15.150:0/1412830028 >> v2:172.21.15.187:3300/0 conn(0x7f3d040b3e10 0x7f3d040cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.056+0000 7f3d09200640 1 -- 172.21.15.150:0/1412830028 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d040b4620 con 0x7f3d041461f0 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.057+0000 7f3d09200640 1 --2- 172.21.15.150:0/1412830028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d041461f0 0x7f3d040ce8b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009870 tx=0x7f3cf802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e251d5db4e38fc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.057+0000 7f3d03400640 1 -- 172.21.15.150:0/1412830028 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8035020 con 0x7f3d041461f0 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.057+0000 7f3d03400640 1 -- 172.21.15.150:0/1412830028 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf8031640 con 0x7f3d041461f0 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.057+0000 7f3d03400640 1 -- 172.21.15.150:0/1412830028 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf802d040 con 0x7f3d041461f0 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 -- 172.21.15.150:0/1412830028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d041461f0 msgr2=0x7f3d040ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/1412830028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d041461f0 0x7f3d040ce8b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009870 tx=0x7f3cf802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 -- 172.21.15.150:0/1412830028 shutdown_connections 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/1412830028 >> v2:172.21.15.187:3300/0 conn(0x7f3d040b3e10 0x7f3d040cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/1412830028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040cedf0 0x7f3d040b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/1412830028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d041461f0 0x7f3d040ce8b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 -- 172.21.15.150:0/1412830028 >> 172.21.15.150:0/1412830028 conn(0x7f3d040b27b0 msgr2=0x7f3d040b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 -- 172.21.15.150:0/1412830028 shutdown_connections 2024-04-15T15:24:05.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.058+0000 7f3d0bc00640 1 -- 172.21.15.150:0/1412830028 wait complete. 2024-04-15T15:24:05.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.060+0000 7f3d0bc00640 1 Processor -- start 2024-04-15T15:24:05.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.060+0000 7f3d0bc00640 1 -- start start 2024-04-15T15:24:05.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.060+0000 7f3d0bc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040b3e10 0x7f3d040cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.060+0000 7f3d0bc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d040cedf0 0x7f3d0415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.060+0000 7f3d0bc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d041461f0 0x7f3d041e7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.060+0000 7f3d0bc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0415d290 con 0x7f3d040cedf0 2024-04-15T15:24:05.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.060+0000 7f3d0bc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0415cfb0 con 0x7f3d041461f0 2024-04-15T15:24:05.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d0bc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d041465d0 con 0x7f3d040b3e10 2024-04-15T15:24:05.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d040cedf0 0x7f3d0415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d040cedf0 0x7f3d0415c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44786/0 (socket says 172.21.15.150:44786) 2024-04-15T15:24:05.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 -- 172.21.15.150:0/3203565354 learned_addr learned my addr 172.21.15.150:0/3203565354 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d09200640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040b3e10 0x7f3d040cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 -- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040b3e10 msgr2=0x7f3d040cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d09c00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d041461f0 0x7f3d041e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040b3e10 0x7f3d040cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 -- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d041461f0 msgr2=0x7f3d041e7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d041461f0 0x7f3d041e7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 -- 172.21.15.150:0/3203565354 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d040cbee0 con 0x7f3d040cedf0 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.061+0000 7f3d03e00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d040cedf0 0x7f3d0415c150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000c7f0 tx=0x7f3cf000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.062+0000 7f3d00c00640 1 -- 172.21.15.150:0/3203565354 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0015070 con 0x7f3d040cedf0 2024-04-15T15:24:05.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.062+0000 7f3d00c00640 1 -- 172.21.15.150:0/3203565354 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf0004240 con 0x7f3d040cedf0 2024-04-15T15:24:05.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.062+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d040cc1c0 con 0x7f3d040cedf0 2024-04-15T15:24:05.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.062+0000 7f3d00c00640 1 -- 172.21.15.150:0/3203565354 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0007620 con 0x7f3d040cedf0 2024-04-15T15:24:05.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.062+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d040cbbf0 con 0x7f3d040cedf0 2024-04-15T15:24:05.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.063+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d040b2dc0 con 0x7f3d040cedf0 2024-04-15T15:24:05.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.064+0000 7f3d00c00640 1 -- 172.21.15.150:0/3203565354 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cf00079d0 con 0x7f3d040cedf0 2024-04-15T15:24:05.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.065+0000 7f3d00c00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ce406c920 0x7f3ce406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.065+0000 7f3d00c00640 1 -- 172.21.15.150:0/3203565354 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3cf008bc00 con 0x7f3d040cedf0 2024-04-15T15:24:05.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.065+0000 7f3d09200640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ce406c920 0x7f3ce406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.067+0000 7f3d09200640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ce406c920 0x7f3ce406ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8036000 tx=0x7f3cf803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:05.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.069+0000 7f3d00c00640 1 -- 172.21.15.150:0/3203565354 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d040b2dc0 con 0x7f3d040cedf0 2024-04-15T15:24:05.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.253+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d041e4e60 con 0x7f3ce406c920 2024-04-15T15:24:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.272+0000 7f3d00c00640 1 -- 172.21.15.150:0/3203565354 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f3d041e4e60 con 0x7f3ce406c920 2024-04-15T15:24:05.277 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:05.279 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:24:05.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ce406c920 msgr2=0x7f3ce406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ce406c920 0x7f3ce406ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8036000 tx=0x7f3cf803c040 comp rx=0 tx=0).stop 2024-04-15T15:24:05.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d040cedf0 msgr2=0x7f3d0415c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d040cedf0 0x7f3d0415c150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000c7f0 tx=0x7f3cf000ccb0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 shutdown_connections 2024-04-15T15:24:05.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d041461f0 0x7f3d041e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d040cedf0 0x7f3d0415c150 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ce406c920 0x7f3ce406ede0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 --2- 172.21.15.150:0/3203565354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d040b3e10 0x7f3d040cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 >> 172.21.15.150:0/3203565354 conn(0x7f3d040b27b0 msgr2=0x7f3d0415f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:05.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 shutdown_connections 2024-04-15T15:24:05.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.278+0000 7f3d0bc00640 1 -- 172.21.15.150:0/3203565354 wait complete. 2024-04-15T15:24:05.298 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-15T15:24:05.219982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.992919"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418140+0000","last_change":"2024-04-15T15:23:55.263891+0000","last_active":"2024-04-15T15:23:56.418140+0000","last_peered":"2024-04-15T15:23:56.418140+0000","last_clean":"2024-04-15T15:23:56.418140+0000","last_became_active":"2024-04-15T15:23:55.263577+0000","last_became_peered":"2024-04-15T15:23:55.263577+0000","last_unstale":"2024-04-15T15:23:56.418140+0000","last_undegraded":"2024-04-15T15:23:56.418140+0000","last_fullsized":"2024-04-15T15:23:56.418140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:58.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255982+0000","last_change":"2024-04-15T15:23:55.259842+0000","last_active":"2024-04-15T15:23:56.255982+0000","last_peered":"2024-04-15T15:23:56.255982+0000","last_clean":"2024-04-15T15:23:56.255982+0000","last_became_active":"2024-04-15T15:23:55.259375+0000","last_became_peered":"2024-04-15T15:23:55.259375+0000","last_unstale":"2024-04-15T15:23:56.255982+0000","last_undegraded":"2024-04-15T15:23:56.255982+0000","last_fullsized":"2024-04-15T15:23:56.255982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418953+0000","last_change":"2024-04-15T15:23:55.249464+0000","last_active":"2024-04-15T15:23:56.418953+0000","last_peered":"2024-04-15T15:23:56.418953+0000","last_clean":"2024-04-15T15:23:56.418953+0000","last_became_active":"2024-04-15T15:23:55.248697+0000","last_became_peered":"2024-04-15T15:23:55.248697+0000","last_unstale":"2024-04-15T15:23:56.418953+0000","last_undegraded":"2024-04-15T15:23:56.418953+0000","last_fullsized":"2024-04-15T15:23:56.418953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262156+0000","last_change":"2024-04-15T15:23:55.267985+0000","last_active":"2024-04-15T15:23:56.262156+0000","last_peered":"2024-04-15T15:23:56.262156+0000","last_clean":"2024-04-15T15:23:56.262156+0000","last_became_active":"2024-04-15T15:23:55.267575+0000","last_became_peered":"2024-04-15T15:23:55.267575+0000","last_unstale":"2024-04-15T15:23:56.262156+0000","last_undegraded":"2024-04-15T15:23:56.262156+0000","last_fullsized":"2024-04-15T15:23:56.262156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:11.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419665+0000","last_change":"2024-04-15T15:23:55.260235+0000","last_active":"2024-04-15T15:23:56.419665+0000","last_peered":"2024-04-15T15:23:56.419665+0000","last_clean":"2024-04-15T15:23:56.419665+0000","last_became_active":"2024-04-15T15:23:55.259947+0000","last_became_peered":"2024-04-15T15:23:55.259947+0000","last_unstale":"2024-04-15T15:23:56.419665+0000","last_undegraded":"2024-04-15T15:23:56.419665+0000","last_fullsized":"2024-04-15T15:23:56.419665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:18:41.223608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261963+0000","last_change":"2024-04-15T15:23:55.257936+0000","last_active":"2024-04-15T15:23:56.261963+0000","last_peered":"2024-04-15T15:23:56.261963+0000","last_clean":"2024-04-15T15:23:56.261963+0000","last_became_active":"2024-04-15T15:23:55.257397+0000","last_became_peered":"2024-04-15T15:23:55.257397+0000","last_unstale":"2024-04-15T15:23:56.261963+0000","last_undegraded":"2024-04-15T15:23:56.261963+0000","last_fullsized":"2024-04-15T15:23:56.261963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417763+0000","last_change":"2024-04-15T15:23:55.265207+0000","last_active":"2024-04-15T15:23:56.417763+0000","last_peered":"2024-04-15T15:23:56.417763+0000","last_clean":"2024-04-15T15:23:56.417763+0000","last_became_active":"2024-04-15T15:23:55.264857+0000","last_became_peered":"2024-04-15T15:23:55.264857+0000","last_unstale":"2024-04-15T15:23:56.417763+0000","last_undegraded":"2024-04-15T15:23:56.417763+0000","last_fullsized":"2024-04-15T15:23:56.417763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:35.342060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416683+0000","last_change":"2024-04-15T15:23:55.270590+0000","last_active":"2024-04-15T15:23:56.416683+0000","last_peered":"2024-04-15T15:23:56.416683+0000","last_clean":"2024-04-15T15:23:56.416683+0000","last_became_active":"2024-04-15T15:23:55.267976+0000","last_became_peered":"2024-04-15T15:23:55.267976+0000","last_unstale":"2024-04-15T15:23:56.416683+0000","last_undegraded":"2024-04-15T15:23:56.416683+0000","last_fullsized":"2024-04-15T15:23:56.416683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:11.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418517+0000","last_change":"2024-04-15T15:23:55.274056+0000","last_active":"2024-04-15T15:23:56.418517+0000","last_peered":"2024-04-15T15:23:56.418517+0000","last_clean":"2024-04-15T15:23:56.418517+0000","last_became_active":"2024-04-15T15:23:55.273746+0000","last_became_peered":"2024-04-15T15:23:55.273746+0000","last_unstale":"2024-04-15T15:23:56.418517+0000","last_undegraded":"2024-04-15T15:23:56.418517+0000","last_fullsized":"2024-04-15T15:23:56.418517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:31:11.922705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262425+0000","last_change":"2024-04-15T15:23:55.272281+0000","last_active":"2024-04-15T15:23:56.262425+0000","last_peered":"2024-04-15T15:23:56.262425+0000","last_clean":"2024-04-15T15:23:56.262425+0000","last_became_active":"2024-04-15T15:23:55.269795+0000","last_became_peered":"2024-04-15T15:23:55.269795+0000","last_unstale":"2024-04-15T15:23:56.262425+0000","last_undegraded":"2024-04-15T15:23:56.262425+0000","last_fullsized":"2024-04-15T15:23:56.262425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263663+0000","last_change":"2024-04-15T15:23:55.269199+0000","last_active":"2024-04-15T15:23:56.263663+0000","last_peered":"2024-04-15T15:23:56.263663+0000","last_clean":"2024-04-15T15:23:56.263663+0000","last_became_active":"2024-04-15T15:23:55.268870+0000","last_became_peered":"2024-04-15T15:23:55.268870+0000","last_unstale":"2024-04-15T15:23:56.263663+0000","last_undegraded":"2024-04-15T15:23:56.263663+0000","last_fullsized":"2024-04-15T15:23:56.263663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418307+0000","last_change":"2024-04-15T15:23:55.263524+0000","last_active":"2024-04-15T15:23:56.418307+0000","last_peered":"2024-04-15T15:23:56.418307+0000","last_clean":"2024-04-15T15:23:56.418307+0000","last_became_active":"2024-04-15T15:23:55.263210+0000","last_became_peered":"2024-04-15T15:23:55.263210+0000","last_unstale":"2024-04-15T15:23:56.418307+0000","last_undegraded":"2024-04-15T15:23:56.418307+0000","last_fullsized":"2024-04-15T15:23:56.418307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:32.154491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415704+0000","last_change":"2024-04-15T15:23:55.272996+0000","last_active":"2024-04-15T15:23:56.415704+0000","last_peered":"2024-04-15T15:23:56.415704+0000","last_clean":"2024-04-15T15:23:56.415704+0000","last_became_active":"2024-04-15T15:23:55.272701+0000","last_became_peered":"2024-04-15T15:23:55.272701+0000","last_unstale":"2024-04-15T15:23:56.415704+0000","last_undegraded":"2024-04-15T15:23:56.415704+0000","last_fullsized":"2024-04-15T15:23:56.415704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417462+0000","last_change":"2024-04-15T15:23:55.260534+0000","last_active":"2024-04-15T15:23:56.417462+0000","last_peered":"2024-04-15T15:23:56.417462+0000","last_clean":"2024-04-15T15:23:56.417462+0000","last_became_active":"2024-04-15T15:23:55.258506+0000","last_became_peered":"2024-04-15T15:23:55.258506+0000","last_unstale":"2024-04-15T15:23:56.417462+0000","last_undegraded":"2024-04-15T15:23:56.417462+0000","last_fullsized":"2024-04-15T15:23:56.417462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255395+0000","last_change":"2024-04-15T15:23:55.265936+0000","last_active":"2024-04-15T15:23:56.255395+0000","last_peered":"2024-04-15T15:23:56.255395+0000","last_clean":"2024-04-15T15:23:56.255395+0000","last_became_active":"2024-04-15T15:23:55.265118+0000","last_became_peered":"2024-04-15T15:23:55.265118+0000","last_unstale":"2024-04-15T15:23:56.255395+0000","last_undegraded":"2024-04-15T15:23:56.255395+0000","last_fullsized":"2024-04-15T15:23:56.255395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.260985+0000","last_change":"2024-04-15T15:23:55.252289+0000","last_active":"2024-04-15T15:23:56.260985+0000","last_peered":"2024-04-15T15:23:56.260985+0000","last_clean":"2024-04-15T15:23:56.260985+0000","last_became_active":"2024-04-15T15:23:55.252007+0000","last_became_peered":"2024-04-15T15:23:55.252007+0000","last_unstale":"2024-04-15T15:23:56.260985+0000","last_undegraded":"2024-04-15T15:23:56.260985+0000","last_fullsized":"2024-04-15T15:23:56.260985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262956+0000","last_change":"2024-04-15T15:23:55.256170+0000","last_active":"2024-04-15T15:23:56.262956+0000","last_peered":"2024-04-15T15:23:56.262956+0000","last_clean":"2024-04-15T15:23:56.262956+0000","last_became_active":"2024-04-15T15:23:55.255896+0000","last_became_peered":"2024-04-15T15:23:55.255896+0000","last_unstale":"2024-04-15T15:23:56.262956+0000","last_undegraded":"2024-04-15T15:23:56.262956+0000","last_fullsized":"2024-04-15T15:23:56.262956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:16:50.418929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417610+0000","last_change":"2024-04-15T15:23:55.269766+0000","last_active":"2024-04-15T15:23:56.417610+0000","last_peered":"2024-04-15T15:23:56.417610+0000","last_clean":"2024-04-15T15:23:56.417610+0000","last_became_active":"2024-04-15T15:23:55.269443+0000","last_became_peered":"2024-04-15T15:23:55.269443+0000","last_unstale":"2024-04-15T15:23:56.417610+0000","last_undegraded":"2024-04-15T15:23:56.417610+0000","last_fullsized":"2024-04-15T15:23:56.417610+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:43:55.932310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419019+0000","last_change":"2024-04-15T15:23:55.265016+0000","last_active":"2024-04-15T15:23:56.419019+0000","last_peered":"2024-04-15T15:23:56.419019+0000","last_clean":"2024-04-15T15:23:56.419019+0000","last_became_active":"2024-04-15T15:23:55.258660+0000","last_became_peered":"2024-04-15T15:23:55.258660+0000","last_unstale":"2024-04-15T15:23:56.419019+0000","last_undegraded":"2024-04-15T15:23:56.419019+0000","last_fullsized":"2024-04-15T15:23:56.419019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:08:08.360698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262025+0000","last_change":"2024-04-15T15:23:55.275816+0000","last_active":"2024-04-15T15:23:56.262025+0000","last_peered":"2024-04-15T15:23:56.262025+0000","last_clean":"2024-04-15T15:23:56.262025+0000","last_became_active":"2024-04-15T15:23:55.274960+0000","last_became_peered":"2024-04-15T15:23:55.274960+0000","last_unstale":"2024-04-15T15:23:56.262025+0000","last_undegraded":"2024-04-15T15:23:56.262025+0000","last_fullsized":"2024-04-15T15:23:56.262025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417084+0000","last_change":"2024-04-15T15:23:55.267469+0000","last_active":"2024-04-15T15:23:56.417084+0000","last_peered":"2024-04-15T15:23:56.417084+0000","last_clean":"2024-04-15T15:23:56.417084+0000","last_became_active":"2024-04-15T15:23:55.266878+0000","last_became_peered":"2024-04-15T15:23:55.266878+0000","last_unstale":"2024-04-15T15:23:56.417084+0000","last_undegraded":"2024-04-15T15:23:56.417084+0000","last_fullsized":"2024-04-15T15:23:56.417084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262363+0000","last_change":"2024-04-15T15:23:55.262231+0000","last_active":"2024-04-15T15:23:56.262363+0000","last_peered":"2024-04-15T15:23:56.262363+0000","last_clean":"2024-04-15T15:23:56.262363+0000","last_became_active":"2024-04-15T15:23:55.261861+0000","last_became_peered":"2024-04-15T15:23:55.261861+0000","last_unstale":"2024-04-15T15:23:56.262363+0000","last_undegraded":"2024-04-15T15:23:56.262363+0000","last_fullsized":"2024-04-15T15:23:56.262363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:38.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417901+0000","last_change":"2024-04-15T15:23:55.275456+0000","last_active":"2024-04-15T15:23:56.417901+0000","last_peered":"2024-04-15T15:23:56.417901+0000","last_clean":"2024-04-15T15:23:56.417901+0000","last_became_active":"2024-04-15T15:23:55.275108+0000","last_became_peered":"2024-04-15T15:23:55.275108+0000","last_unstale":"2024-04-15T15:23:56.417901+0000","last_undegraded":"2024-04-15T15:23:56.417901+0000","last_fullsized":"2024-04-15T15:23:56.417901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:38.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261274+0000","last_change":"2024-04-15T15:23:55.270445+0000","last_active":"2024-04-15T15:23:56.261274+0000","last_peered":"2024-04-15T15:23:56.261274+0000","last_clean":"2024-04-15T15:23:56.261274+0000","last_became_active":"2024-04-15T15:23:55.270131+0000","last_became_peered":"2024-04-15T15:23:55.270131+0000","last_unstale":"2024-04-15T15:23:56.261274+0000","last_undegraded":"2024-04-15T15:23:56.261274+0000","last_fullsized":"2024-04-15T15:23:56.261274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415461+0000","last_change":"2024-04-15T15:23:55.260659+0000","last_active":"2024-04-15T15:23:56.415461+0000","last_peered":"2024-04-15T15:23:56.415461+0000","last_clean":"2024-04-15T15:23:56.415461+0000","last_became_active":"2024-04-15T15:23:55.258739+0000","last_became_peered":"2024-04-15T15:23:55.258739+0000","last_unstale":"2024-04-15T15:23:56.415461+0000","last_undegraded":"2024-04-15T15:23:56.415461+0000","last_fullsized":"2024-04-15T15:23:56.415461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262634+0000","last_change":"2024-04-15T15:23:55.272818+0000","last_active":"2024-04-15T15:23:56.262634+0000","last_peered":"2024-04-15T15:23:56.262634+0000","last_clean":"2024-04-15T15:23:56.262634+0000","last_became_active":"2024-04-15T15:23:55.269858+0000","last_became_peered":"2024-04-15T15:23:55.269858+0000","last_unstale":"2024-04-15T15:23:56.262634+0000","last_undegraded":"2024-04-15T15:23:56.262634+0000","last_fullsized":"2024-04-15T15:23:56.262634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255726+0000","last_change":"2024-04-15T15:23:55.261128+0000","last_active":"2024-04-15T15:23:56.255726+0000","last_peered":"2024-04-15T15:23:56.255726+0000","last_clean":"2024-04-15T15:23:56.255726+0000","last_became_active":"2024-04-15T15:23:55.259248+0000","last_became_peered":"2024-04-15T15:23:55.259248+0000","last_unstale":"2024-04-15T15:23:56.255726+0000","last_undegraded":"2024-04-15T15:23:56.255726+0000","last_fullsized":"2024-04-15T15:23:56.255726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415563+0000","last_change":"2024-04-15T15:23:55.264402+0000","last_active":"2024-04-15T15:23:56.415563+0000","last_peered":"2024-04-15T15:23:56.415563+0000","last_clean":"2024-04-15T15:23:56.415563+0000","last_became_active":"2024-04-15T15:23:55.264083+0000","last_became_peered":"2024-04-15T15:23:55.264083+0000","last_unstale":"2024-04-15T15:23:56.415563+0000","last_undegraded":"2024-04-15T15:23:56.415563+0000","last_fullsized":"2024-04-15T15:23:56.415563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417369+0000","last_change":"2024-04-15T15:23:55.262147+0000","last_active":"2024-04-15T15:23:56.417369+0000","last_peered":"2024-04-15T15:23:56.417369+0000","last_clean":"2024-04-15T15:23:56.417369+0000","last_became_active":"2024-04-15T15:23:55.261524+0000","last_became_peered":"2024-04-15T15:23:55.261524+0000","last_unstale":"2024-04-15T15:23:56.417369+0000","last_undegraded":"2024-04-15T15:23:56.417369+0000","last_fullsized":"2024-04-15T15:23:56.417369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262811+0000","last_change":"2024-04-15T15:23:55.267385+0000","last_active":"2024-04-15T15:23:56.262811+0000","last_peered":"2024-04-15T15:23:56.262811+0000","last_clean":"2024-04-15T15:23:56.262811+0000","last_became_active":"2024-04-15T15:23:55.267045+0000","last_became_peered":"2024-04-15T15:23:55.267045+0000","last_unstale":"2024-04-15T15:23:56.262811+0000","last_undegraded":"2024-04-15T15:23:56.262811+0000","last_fullsized":"2024-04-15T15:23:56.262811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:35.342060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417143+0000","last_change":"2024-04-15T15:23:55.261039+0000","last_active":"2024-04-15T15:23:56.417143+0000","last_peered":"2024-04-15T15:23:56.417143+0000","last_clean":"2024-04-15T15:23:56.417143+0000","last_became_active":"2024-04-15T15:23:55.260620+0000","last_became_peered":"2024-04-15T15:23:55.260620+0000","last_unstale":"2024-04-15T15:23:56.417143+0000","last_undegraded":"2024-04-15T15:23:56.417143+0000","last_fullsized":"2024-04-15T15:23:56.417143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:27:44.245014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261366+0000","last_change":"2024-04-15T15:23:55.271261+0000","last_active":"2024-04-15T15:23:56.261366+0000","last_peered":"2024-04-15T15:23:56.261366+0000","last_clean":"2024-04-15T15:23:56.261366+0000","last_became_active":"2024-04-15T15:23:55.270954+0000","last_became_peered":"2024-04-15T15:23:55.270954+0000","last_unstale":"2024-04-15T15:23:56.261366+0000","last_undegraded":"2024-04-15T15:23:56.261366+0000","last_fullsized":"2024-04-15T15:23:56.261366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417469+0000","last_change":"2024-04-15T15:23:55.263715+0000","last_active":"2024-04-15T15:23:56.417469+0000","last_peered":"2024-04-15T15:23:56.417469+0000","last_clean":"2024-04-15T15:23:56.417469+0000","last_became_active":"2024-04-15T15:23:55.256992+0000","last_became_peered":"2024-04-15T15:23:55.256992+0000","last_unstale":"2024-04-15T15:23:56.417469+0000","last_undegraded":"2024-04-15T15:23:56.417469+0000","last_fullsized":"2024-04-15T15:23:56.417469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:25.936170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416848+0000","last_change":"2024-04-15T15:23:54.250784+0000","last_active":"2024-04-15T15:23:56.416848+0000","last_peered":"2024-04-15T15:23:56.416848+0000","last_clean":"2024-04-15T15:23:56.416848+0000","last_became_active":"2024-04-15T15:23:54.250347+0000","last_became_peered":"2024-04-15T15:23:54.250347+0000","last_unstale":"2024-04-15T15:23:56.416848+0000","last_undegraded":"2024-04-15T15:23:56.416848+0000","last_fullsized":"2024-04-15T15:23:56.416848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417629+0000","last_change":"2024-04-15T15:23:55.262408+0000","last_active":"2024-04-15T15:23:56.417629+0000","last_peered":"2024-04-15T15:23:56.417629+0000","last_clean":"2024-04-15T15:23:56.417629+0000","last_became_active":"2024-04-15T15:23:55.261815+0000","last_became_peered":"2024-04-15T15:23:55.261815+0000","last_unstale":"2024-04-15T15:23:56.417629+0000","last_undegraded":"2024-04-15T15:23:56.417629+0000","last_fullsized":"2024-04-15T15:23:56.417629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261683+0000","last_change":"2024-04-15T15:23:54.239676+0000","last_active":"2024-04-15T15:23:56.261683+0000","last_peered":"2024-04-15T15:23:56.261683+0000","last_clean":"2024-04-15T15:23:56.261683+0000","last_became_active":"2024-04-15T15:23:54.239084+0000","last_became_peered":"2024-04-15T15:23:54.239084+0000","last_unstale":"2024-04-15T15:23:56.261683+0000","last_undegraded":"2024-04-15T15:23:56.261683+0000","last_fullsized":"2024-04-15T15:23:56.261683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415034+0000","last_change":"2024-04-15T15:23:54.241378+0000","last_active":"2024-04-15T15:23:56.415034+0000","last_peered":"2024-04-15T15:23:56.415034+0000","last_clean":"2024-04-15T15:23:56.415034+0000","last_became_active":"2024-04-15T15:23:54.241084+0000","last_became_peered":"2024-04-15T15:23:54.241084+0000","last_unstale":"2024-04-15T15:23:56.415034+0000","last_undegraded":"2024-04-15T15:23:56.415034+0000","last_fullsized":"2024-04-15T15:23:56.415034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:38:14.692145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.414885+0000","last_change":"2024-04-15T15:23:55.257791+0000","last_active":"2024-04-15T15:23:56.414885+0000","last_peered":"2024-04-15T15:23:56.414885+0000","last_clean":"2024-04-15T15:23:56.414885+0000","last_became_active":"2024-04-15T15:23:55.257531+0000","last_became_peered":"2024-04-15T15:23:55.257531+0000","last_unstale":"2024-04-15T15:23:56.414885+0000","last_undegraded":"2024-04-15T15:23:56.414885+0000","last_fullsized":"2024-04-15T15:23:56.414885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261118+0000","last_change":"2024-04-15T15:23:54.235199+0000","last_active":"2024-04-15T15:23:56.261118+0000","last_peered":"2024-04-15T15:23:56.261118+0000","last_clean":"2024-04-15T15:23:56.261118+0000","last_became_active":"2024-04-15T15:23:54.233620+0000","last_became_peered":"2024-04-15T15:23:54.233620+0000","last_unstale":"2024-04-15T15:23:56.261118+0000","last_undegraded":"2024-04-15T15:23:56.261118+0000","last_fullsized":"2024-04-15T15:23:56.261118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255313+0000","last_change":"2024-04-15T15:23:55.261632+0000","last_active":"2024-04-15T15:23:56.255313+0000","last_peered":"2024-04-15T15:23:56.255313+0000","last_clean":"2024-04-15T15:23:56.255313+0000","last_became_active":"2024-04-15T15:23:55.261351+0000","last_became_peered":"2024-04-15T15:23:55.261351+0000","last_unstale":"2024-04-15T15:23:56.255313+0000","last_undegraded":"2024-04-15T15:23:56.255313+0000","last_fullsized":"2024-04-15T15:23:56.255313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263032+0000","last_change":"2024-04-15T15:23:54.237429+0000","last_active":"2024-04-15T15:23:56.263032+0000","last_peered":"2024-04-15T15:23:56.263032+0000","last_clean":"2024-04-15T15:23:56.263032+0000","last_became_active":"2024-04-15T15:23:54.237045+0000","last_became_peered":"2024-04-15T15:23:54.237045+0000","last_unstale":"2024-04-15T15:23:56.263032+0000","last_undegraded":"2024-04-15T15:23:56.263032+0000","last_fullsized":"2024-04-15T15:23:56.263032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262636+0000","last_change":"2024-04-15T15:23:55.263292+0000","last_active":"2024-04-15T15:23:56.262636+0000","last_peered":"2024-04-15T15:23:56.262636+0000","last_clean":"2024-04-15T15:23:56.262636+0000","last_became_active":"2024-04-15T15:23:55.261899+0000","last_became_peered":"2024-04-15T15:23:55.261899+0000","last_unstale":"2024-04-15T15:23:56.262636+0000","last_undegraded":"2024-04-15T15:23:56.262636+0000","last_fullsized":"2024-04-15T15:23:56.262636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:40.716704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262724+0000","last_change":"2024-04-15T15:23:54.242528+0000","last_active":"2024-04-15T15:23:56.262724+0000","last_peered":"2024-04-15T15:23:56.262724+0000","last_clean":"2024-04-15T15:23:56.262724+0000","last_became_active":"2024-04-15T15:23:54.242233+0000","last_became_peered":"2024-04-15T15:23:54.242233+0000","last_unstale":"2024-04-15T15:23:56.262724+0000","last_undegraded":"2024-04-15T15:23:56.262724+0000","last_fullsized":"2024-04-15T15:23:56.262724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417168+0000","last_change":"2024-04-15T15:23:55.268964+0000","last_active":"2024-04-15T15:23:56.417168+0000","last_peered":"2024-04-15T15:23:56.417168+0000","last_clean":"2024-04-15T15:23:56.417168+0000","last_became_active":"2024-04-15T15:23:55.268348+0000","last_became_peered":"2024-04-15T15:23:55.268348+0000","last_unstale":"2024-04-15T15:23:56.417168+0000","last_undegraded":"2024-04-15T15:23:56.417168+0000","last_fullsized":"2024-04-15T15:23:56.417168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255623+0000","last_change":"2024-04-15T15:23:54.238578+0000","last_active":"2024-04-15T15:23:56.255623+0000","last_peered":"2024-04-15T15:23:56.255623+0000","last_clean":"2024-04-15T15:23:56.255623+0000","last_became_active":"2024-04-15T15:23:54.238332+0000","last_became_peered":"2024-04-15T15:23:54.238332+0000","last_unstale":"2024-04-15T15:23:56.255623+0000","last_undegraded":"2024-04-15T15:23:56.255623+0000","last_fullsized":"2024-04-15T15:23:56.255623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418641+0000","last_change":"2024-04-15T15:23:55.266173+0000","last_active":"2024-04-15T15:23:56.418641+0000","last_peered":"2024-04-15T15:23:56.418641+0000","last_clean":"2024-04-15T15:23:56.418641+0000","last_became_active":"2024-04-15T15:23:55.265877+0000","last_became_peered":"2024-04-15T15:23:55.265877+0000","last_unstale":"2024-04-15T15:23:56.418641+0000","last_undegraded":"2024-04-15T15:23:56.418641+0000","last_fullsized":"2024-04-15T15:23:56.418641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:52.386885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262108+0000","last_change":"2024-04-15T15:23:54.240249+0000","last_active":"2024-04-15T15:23:56.262108+0000","last_peered":"2024-04-15T15:23:56.262108+0000","last_clean":"2024-04-15T15:23:56.262108+0000","last_became_active":"2024-04-15T15:23:54.239151+0000","last_became_peered":"2024-04-15T15:23:54.239151+0000","last_unstale":"2024-04-15T15:23:56.262108+0000","last_undegraded":"2024-04-15T15:23:56.262108+0000","last_fullsized":"2024-04-15T15:23:56.262108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417969+0000","last_change":"2024-04-15T15:23:55.266293+0000","last_active":"2024-04-15T15:23:56.417969+0000","last_peered":"2024-04-15T15:23:56.417969+0000","last_clean":"2024-04-15T15:23:56.417969+0000","last_became_active":"2024-04-15T15:23:55.265962+0000","last_became_peered":"2024-04-15T15:23:55.265962+0000","last_unstale":"2024-04-15T15:23:56.417969+0000","last_undegraded":"2024-04-15T15:23:56.417969+0000","last_fullsized":"2024-04-15T15:23:56.417969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:40.334427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417723+0000","last_change":"2024-04-15T15:23:54.233459+0000","last_active":"2024-04-15T15:23:56.417723+0000","last_peered":"2024-04-15T15:23:56.417723+0000","last_clean":"2024-04-15T15:23:56.417723+0000","last_became_active":"2024-04-15T15:23:54.233056+0000","last_became_peered":"2024-04-15T15:23:54.233056+0000","last_unstale":"2024-04-15T15:23:56.417723+0000","last_undegraded":"2024-04-15T15:23:56.417723+0000","last_fullsized":"2024-04-15T15:23:56.417723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:38:14.692145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417725+0000","last_change":"2024-04-15T15:23:55.261166+0000","last_active":"2024-04-15T15:23:56.417725+0000","last_peered":"2024-04-15T15:23:56.417725+0000","last_clean":"2024-04-15T15:23:56.417725+0000","last_became_active":"2024-04-15T15:23:55.260827+0000","last_became_peered":"2024-04-15T15:23:55.260827+0000","last_unstale":"2024-04-15T15:23:56.417725+0000","last_undegraded":"2024-04-15T15:23:56.417725+0000","last_fullsized":"2024-04-15T15:23:56.417725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262944+0000","last_change":"2024-04-15T15:23:54.233681+0000","last_active":"2024-04-15T15:23:56.262944+0000","last_peered":"2024-04-15T15:23:56.262944+0000","last_clean":"2024-04-15T15:23:56.262944+0000","last_became_active":"2024-04-15T15:23:54.233376+0000","last_became_peered":"2024-04-15T15:23:54.233376+0000","last_unstale":"2024-04-15T15:23:56.262944+0000","last_undegraded":"2024-04-15T15:23:56.262944+0000","last_fullsized":"2024-04-15T15:23:56.262944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419714+0000","last_change":"2024-04-15T15:23:55.275827+0000","last_active":"2024-04-15T15:23:56.419714+0000","last_peered":"2024-04-15T15:23:56.419714+0000","last_clean":"2024-04-15T15:23:56.419714+0000","last_became_active":"2024-04-15T15:23:55.275511+0000","last_became_peered":"2024-04-15T15:23:55.275511+0000","last_unstale":"2024-04-15T15:23:56.419714+0000","last_undegraded":"2024-04-15T15:23:56.419714+0000","last_fullsized":"2024-04-15T15:23:56.419714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:17:08.438022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261296+0000","last_change":"2024-04-15T15:23:54.237666+0000","last_active":"2024-04-15T15:23:56.261296+0000","last_peered":"2024-04-15T15:23:56.261296+0000","last_clean":"2024-04-15T15:23:56.261296+0000","last_became_active":"2024-04-15T15:23:54.237243+0000","last_became_peered":"2024-04-15T15:23:54.237243+0000","last_unstale":"2024-04-15T15:23:56.261296+0000","last_undegraded":"2024-04-15T15:23:56.261296+0000","last_fullsized":"2024-04-15T15:23:56.261296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262905+0000","last_change":"2024-04-15T15:23:55.264795+0000","last_active":"2024-04-15T15:23:56.262905+0000","last_peered":"2024-04-15T15:23:56.262905+0000","last_clean":"2024-04-15T15:23:56.262905+0000","last_became_active":"2024-04-15T15:23:55.264509+0000","last_became_peered":"2024-04-15T15:23:55.264509+0000","last_unstale":"2024-04-15T15:23:56.262905+0000","last_undegraded":"2024-04-15T15:23:56.262905+0000","last_fullsized":"2024-04-15T15:23:56.262905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416505+0000","last_change":"2024-04-15T15:23:54.233260+0000","last_active":"2024-04-15T15:23:56.416505+0000","last_peered":"2024-04-15T15:23:56.416505+0000","last_clean":"2024-04-15T15:23:56.416505+0000","last_became_active":"2024-04-15T15:23:54.231670+0000","last_became_peered":"2024-04-15T15:23:54.231670+0000","last_unstale":"2024-04-15T15:23:56.416505+0000","last_undegraded":"2024-04-15T15:23:56.416505+0000","last_fullsized":"2024-04-15T15:23:56.416505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:33:24.512549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418111+0000","last_change":"2024-04-15T15:23:55.278880+0000","last_active":"2024-04-15T15:23:56.418111+0000","last_peered":"2024-04-15T15:23:56.418111+0000","last_clean":"2024-04-15T15:23:56.418111+0000","last_became_active":"2024-04-15T15:23:55.278617+0000","last_became_peered":"2024-04-15T15:23:55.278617+0000","last_unstale":"2024-04-15T15:23:56.418111+0000","last_undegraded":"2024-04-15T15:23:56.418111+0000","last_fullsized":"2024-04-15T15:23:56.418111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:16:50.418929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418140+0000","last_change":"2024-04-15T15:23:54.239459+0000","last_active":"2024-04-15T15:23:56.418140+0000","last_peered":"2024-04-15T15:23:56.418140+0000","last_clean":"2024-04-15T15:23:56.418140+0000","last_became_active":"2024-04-15T15:23:54.239033+0000","last_became_peered":"2024-04-15T15:23:54.239033+0000","last_unstale":"2024-04-15T15:23:56.418140+0000","last_undegraded":"2024-04-15T15:23:56.418140+0000","last_fullsized":"2024-04-15T15:23:56.418140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:16.311793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419071+0000","last_change":"2024-04-15T15:23:55.277191+0000","last_active":"2024-04-15T15:23:56.419071+0000","last_peered":"2024-04-15T15:23:56.419071+0000","last_clean":"2024-04-15T15:23:56.419071+0000","last_became_active":"2024-04-15T15:23:55.276803+0000","last_became_peered":"2024-04-15T15:23:55.276803+0000","last_unstale":"2024-04-15T15:23:56.419071+0000","last_undegraded":"2024-04-15T15:23:56.419071+0000","last_fullsized":"2024-04-15T15:23:56.419071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:53:48.793142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417566+0000","last_change":"2024-04-15T15:23:54.230441+0000","last_active":"2024-04-15T15:23:56.417566+0000","last_peered":"2024-04-15T15:23:56.417566+0000","last_clean":"2024-04-15T15:23:56.417566+0000","last_became_active":"2024-04-15T15:23:54.230051+0000","last_became_peered":"2024-04-15T15:23:54.230051+0000","last_unstale":"2024-04-15T15:23:56.417566+0000","last_undegraded":"2024-04-15T15:23:56.417566+0000","last_fullsized":"2024-04-15T15:23:56.417566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:29.938678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418932+0000","last_change":"2024-04-15T15:23:55.281762+0000","last_active":"2024-04-15T15:23:56.418932+0000","last_peered":"2024-04-15T15:23:56.418932+0000","last_clean":"2024-04-15T15:23:56.418932+0000","last_became_active":"2024-04-15T15:23:55.281494+0000","last_became_peered":"2024-04-15T15:23:55.281494+0000","last_unstale":"2024-04-15T15:23:56.418932+0000","last_undegraded":"2024-04-15T15:23:56.418932+0000","last_fullsized":"2024-04-15T15:23:56.418932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:42:09.480734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255780+0000","last_change":"2024-04-15T15:23:54.234786+0000","last_active":"2024-04-15T15:23:56.255780+0000","last_peered":"2024-04-15T15:23:56.255780+0000","last_clean":"2024-04-15T15:23:56.255780+0000","last_became_active":"2024-04-15T15:23:54.234484+0000","last_became_peered":"2024-04-15T15:23:54.234484+0000","last_unstale":"2024-04-15T15:23:56.255780+0000","last_undegraded":"2024-04-15T15:23:56.255780+0000","last_fullsized":"2024-04-15T15:23:56.255780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:05.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262958+0000","last_change":"2024-04-15T15:23:55.267889+0000","last_active":"2024-04-15T15:23:56.262958+0000","last_peered":"2024-04-15T15:23:56.262958+0000","last_clean":"2024-04-15T15:23:56.262958+0000","last_became_active":"2024-04-15T15:23:55.267209+0000","last_became_peered":"2024-04-15T15:23:55.267209+0000","last_unstale":"2024-04-15T15:23:56.262958+0000","last_undegraded":"2024-04-15T15:23:56.262958+0000","last_fullsized":"2024-04-15T15:23:56.262958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:40.334427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416588+0000","last_change":"2024-04-15T15:23:54.237963+0000","last_active":"2024-04-15T15:23:56.416588+0000","last_peered":"2024-04-15T15:23:56.416588+0000","last_clean":"2024-04-15T15:23:56.416588+0000","last_became_active":"2024-04-15T15:23:54.237726+0000","last_became_peered":"2024-04-15T15:23:54.237726+0000","last_unstale":"2024-04-15T15:23:56.416588+0000","last_undegraded":"2024-04-15T15:23:56.416588+0000","last_fullsized":"2024-04-15T15:23:56.416588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:45:06.635874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418333+0000","last_change":"2024-04-15T15:23:55.272911+0000","last_active":"2024-04-15T15:23:56.418333+0000","last_peered":"2024-04-15T15:23:56.418333+0000","last_clean":"2024-04-15T15:23:56.418333+0000","last_became_active":"2024-04-15T15:23:55.272642+0000","last_became_peered":"2024-04-15T15:23:55.272642+0000","last_unstale":"2024-04-15T15:23:56.418333+0000","last_undegraded":"2024-04-15T15:23:56.418333+0000","last_fullsized":"2024-04-15T15:23:56.418333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:27.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417306+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:23:56.417306+0000","last_peered":"2024-04-15T15:23:56.417306+0000","last_clean":"2024-04-15T15:23:56.417306+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:23:56.417306+0000","last_undegraded":"2024-04-15T15:23:56.417306+0000","last_fullsized":"2024-04-15T15:23:56.417306+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261029+0000","last_change":"2024-04-15T15:23:54.232974+0000","last_active":"2024-04-15T15:23:56.261029+0000","last_peered":"2024-04-15T15:23:56.261029+0000","last_clean":"2024-04-15T15:23:56.261029+0000","last_became_active":"2024-04-15T15:23:54.230808+0000","last_became_peered":"2024-04-15T15:23:54.230808+0000","last_unstale":"2024-04-15T15:23:56.261029+0000","last_undegraded":"2024-04-15T15:23:56.261029+0000","last_fullsized":"2024-04-15T15:23:56.261029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:46:30.058785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418655+0000","last_change":"2024-04-15T15:23:55.269058+0000","last_active":"2024-04-15T15:23:56.418655+0000","last_peered":"2024-04-15T15:23:56.418655+0000","last_clean":"2024-04-15T15:23:56.418655+0000","last_became_active":"2024-04-15T15:23:55.268267+0000","last_became_peered":"2024-04-15T15:23:55.268267+0000","last_unstale":"2024-04-15T15:23:56.418655+0000","last_undegraded":"2024-04-15T15:23:56.418655+0000","last_fullsized":"2024-04-15T15:23:56.418655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:30:18.931762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417047+0000","last_change":"2024-04-15T15:23:54.251259+0000","last_active":"2024-04-15T15:23:56.417047+0000","last_peered":"2024-04-15T15:23:56.417047+0000","last_clean":"2024-04-15T15:23:56.417047+0000","last_became_active":"2024-04-15T15:23:54.250985+0000","last_became_peered":"2024-04-15T15:23:54.250985+0000","last_unstale":"2024-04-15T15:23:56.417047+0000","last_undegraded":"2024-04-15T15:23:56.417047+0000","last_fullsized":"2024-04-15T15:23:56.417047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:44:11.419953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418428+0000","last_change":"2024-04-15T15:23:55.267279+0000","last_active":"2024-04-15T15:23:56.418428+0000","last_peered":"2024-04-15T15:23:56.418428+0000","last_clean":"2024-04-15T15:23:56.418428+0000","last_became_active":"2024-04-15T15:23:55.266971+0000","last_became_peered":"2024-04-15T15:23:55.266971+0000","last_unstale":"2024-04-15T15:23:56.418428+0000","last_undegraded":"2024-04-15T15:23:56.418428+0000","last_fullsized":"2024-04-15T15:23:56.418428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417318+0000","last_change":"2024-04-15T15:23:54.234761+0000","last_active":"2024-04-15T15:23:56.417318+0000","last_peered":"2024-04-15T15:23:56.417318+0000","last_clean":"2024-04-15T15:23:56.417318+0000","last_became_active":"2024-04-15T15:23:54.234467+0000","last_became_peered":"2024-04-15T15:23:54.234467+0000","last_unstale":"2024-04-15T15:23:56.417318+0000","last_undegraded":"2024-04-15T15:23:56.417318+0000","last_fullsized":"2024-04-15T15:23:56.417318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:47:42.194686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255347+0000","last_change":"2024-04-15T15:23:55.244630+0000","last_active":"2024-04-15T15:23:56.255347+0000","last_peered":"2024-04-15T15:23:56.255347+0000","last_clean":"2024-04-15T15:23:56.255347+0000","last_became_active":"2024-04-15T15:23:55.244208+0000","last_became_peered":"2024-04-15T15:23:55.244208+0000","last_unstale":"2024-04-15T15:23:56.255347+0000","last_undegraded":"2024-04-15T15:23:56.255347+0000","last_fullsized":"2024-04-15T15:23:56.255347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263182+0000","last_change":"2024-04-15T15:23:54.225774+0000","last_active":"2024-04-15T15:23:56.263182+0000","last_peered":"2024-04-15T15:23:56.263182+0000","last_clean":"2024-04-15T15:23:56.263182+0000","last_became_active":"2024-04-15T15:23:54.223951+0000","last_became_peered":"2024-04-15T15:23:54.223951+0000","last_unstale":"2024-04-15T15:23:56.263182+0000","last_undegraded":"2024-04-15T15:23:56.263182+0000","last_fullsized":"2024-04-15T15:23:56.263182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417834+0000","last_change":"2024-04-15T15:23:55.261370+0000","last_active":"2024-04-15T15:23:56.417834+0000","last_peered":"2024-04-15T15:23:56.417834+0000","last_clean":"2024-04-15T15:23:56.417834+0000","last_became_active":"2024-04-15T15:23:55.260930+0000","last_became_peered":"2024-04-15T15:23:55.260930+0000","last_unstale":"2024-04-15T15:23:56.417834+0000","last_undegraded":"2024-04-15T15:23:56.417834+0000","last_fullsized":"2024-04-15T15:23:56.417834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:54.270601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255534+0000","last_change":"2024-04-15T15:23:54.223225+0000","last_active":"2024-04-15T15:23:56.255534+0000","last_peered":"2024-04-15T15:23:56.255534+0000","last_clean":"2024-04-15T15:23:56.255534+0000","last_became_active":"2024-04-15T15:23:54.222903+0000","last_became_peered":"2024-04-15T15:23:54.222903+0000","last_unstale":"2024-04-15T15:23:56.255534+0000","last_undegraded":"2024-04-15T15:23:56.255534+0000","last_fullsized":"2024-04-15T15:23:56.255534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:51.381976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.264575+0000","last_change":"2024-04-15T15:23:55.270383+0000","last_active":"2024-04-15T15:23:56.264575+0000","last_peered":"2024-04-15T15:23:56.264575+0000","last_clean":"2024-04-15T15:23:56.264575+0000","last_became_active":"2024-04-15T15:23:55.267157+0000","last_became_peered":"2024-04-15T15:23:55.267157+0000","last_unstale":"2024-04-15T15:23:56.264575+0000","last_undegraded":"2024-04-15T15:23:56.264575+0000","last_fullsized":"2024-04-15T15:23:56.264575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:27.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419141+0000","last_change":"2024-04-15T15:23:54.251497+0000","last_active":"2024-04-15T15:23:56.419141+0000","last_peered":"2024-04-15T15:23:56.419141+0000","last_clean":"2024-04-15T15:23:56.419141+0000","last_became_active":"2024-04-15T15:23:54.250660+0000","last_became_peered":"2024-04-15T15:23:54.250660+0000","last_unstale":"2024-04-15T15:23:56.419141+0000","last_undegraded":"2024-04-15T15:23:56.419141+0000","last_fullsized":"2024-04-15T15:23:56.419141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:14.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255397+0000","last_change":"2024-04-15T15:23:55.249702+0000","last_active":"2024-04-15T15:23:56.255397+0000","last_peered":"2024-04-15T15:23:56.255397+0000","last_clean":"2024-04-15T15:23:56.255397+0000","last_became_active":"2024-04-15T15:23:55.249084+0000","last_became_peered":"2024-04-15T15:23:55.249084+0000","last_unstale":"2024-04-15T15:23:56.255397+0000","last_undegraded":"2024-04-15T15:23:56.255397+0000","last_fullsized":"2024-04-15T15:23:56.255397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255845+0000","last_change":"2024-04-15T15:23:54.241520+0000","last_active":"2024-04-15T15:23:56.255845+0000","last_peered":"2024-04-15T15:23:56.255845+0000","last_clean":"2024-04-15T15:23:56.255845+0000","last_became_active":"2024-04-15T15:23:54.241103+0000","last_became_peered":"2024-04-15T15:23:54.241103+0000","last_unstale":"2024-04-15T15:23:56.255845+0000","last_undegraded":"2024-04-15T15:23:56.255845+0000","last_fullsized":"2024-04-15T15:23:56.255845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:44:11.419953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415819+0000","last_change":"2024-04-15T15:23:55.265567+0000","last_active":"2024-04-15T15:23:56.415819+0000","last_peered":"2024-04-15T15:23:56.415819+0000","last_clean":"2024-04-15T15:23:56.415819+0000","last_became_active":"2024-04-15T15:23:55.262905+0000","last_became_peered":"2024-04-15T15:23:55.262905+0000","last_unstale":"2024-04-15T15:23:56.415819+0000","last_undegraded":"2024-04-15T15:23:56.415819+0000","last_fullsized":"2024-04-15T15:23:56.415819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263461+0000","last_change":"2024-04-15T15:23:54.252083+0000","last_active":"2024-04-15T15:23:56.263461+0000","last_peered":"2024-04-15T15:23:56.263461+0000","last_clean":"2024-04-15T15:23:56.263461+0000","last_became_active":"2024-04-15T15:23:54.251796+0000","last_became_peered":"2024-04-15T15:23:54.251796+0000","last_unstale":"2024-04-15T15:23:56.263461+0000","last_undegraded":"2024-04-15T15:23:56.263461+0000","last_fullsized":"2024-04-15T15:23:56.263461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:46:30.058785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416720+0000","last_change":"2024-04-15T15:23:55.257995+0000","last_active":"2024-04-15T15:23:56.416720+0000","last_peered":"2024-04-15T15:23:56.416720+0000","last_clean":"2024-04-15T15:23:56.416720+0000","last_became_active":"2024-04-15T15:23:55.257718+0000","last_became_peered":"2024-04-15T15:23:55.257718+0000","last_unstale":"2024-04-15T15:23:56.416720+0000","last_undegraded":"2024-04-15T15:23:56.416720+0000","last_fullsized":"2024-04-15T15:23:56.416720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:08:29.731069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416748+0000","last_change":"2024-04-15T15:23:54.250977+0000","last_active":"2024-04-15T15:23:56.416748+0000","last_peered":"2024-04-15T15:23:56.416748+0000","last_clean":"2024-04-15T15:23:56.416748+0000","last_became_active":"2024-04-15T15:23:54.250302+0000","last_became_peered":"2024-04-15T15:23:54.250302+0000","last_unstale":"2024-04-15T15:23:56.416748+0000","last_undegraded":"2024-04-15T15:23:56.416748+0000","last_fullsized":"2024-04-15T15:23:56.416748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:05.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418747+0000","last_change":"2024-04-15T15:23:55.260220+0000","last_active":"2024-04-15T15:23:56.418747+0000","last_peered":"2024-04-15T15:23:56.418747+0000","last_clean":"2024-04-15T15:23:56.418747+0000","last_became_active":"2024-04-15T15:23:55.259361+0000","last_became_peered":"2024-04-15T15:23:55.259361+0000","last_unstale":"2024-04-15T15:23:56.418747+0000","last_undegraded":"2024-04-15T15:23:56.418747+0000","last_fullsized":"2024-04-15T15:23:56.418747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:06.899022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255405+0000","last_change":"2024-04-15T15:23:54.236256+0000","last_active":"2024-04-15T15:23:56.255405+0000","last_peered":"2024-04-15T15:23:56.255405+0000","last_clean":"2024-04-15T15:23:56.255405+0000","last_became_active":"2024-04-15T15:23:54.235919+0000","last_became_peered":"2024-04-15T15:23:54.235919+0000","last_unstale":"2024-04-15T15:23:56.255405+0000","last_undegraded":"2024-04-15T15:23:56.255405+0000","last_fullsized":"2024-04-15T15:23:56.255405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415070+0000","last_change":"2024-04-15T15:23:55.258682+0000","last_active":"2024-04-15T15:23:56.415070+0000","last_peered":"2024-04-15T15:23:56.415070+0000","last_clean":"2024-04-15T15:23:56.415070+0000","last_became_active":"2024-04-15T15:23:55.258183+0000","last_became_peered":"2024-04-15T15:23:55.258183+0000","last_unstale":"2024-04-15T15:23:56.415070+0000","last_undegraded":"2024-04-15T15:23:56.415070+0000","last_fullsized":"2024-04-15T15:23:56.415070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255301+0000","last_change":"2024-04-15T15:23:54.233682+0000","last_active":"2024-04-15T15:23:56.255301+0000","last_peered":"2024-04-15T15:23:56.255301+0000","last_clean":"2024-04-15T15:23:56.255301+0000","last_became_active":"2024-04-15T15:23:54.233373+0000","last_became_peered":"2024-04-15T15:23:54.233373+0000","last_unstale":"2024-04-15T15:23:56.255301+0000","last_undegraded":"2024-04-15T15:23:56.255301+0000","last_fullsized":"2024-04-15T15:23:56.255301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416048+0000","last_change":"2024-04-15T15:23:55.272358+0000","last_active":"2024-04-15T15:23:56.416048+0000","last_peered":"2024-04-15T15:23:56.416048+0000","last_clean":"2024-04-15T15:23:56.416048+0000","last_became_active":"2024-04-15T15:23:55.271660+0000","last_became_peered":"2024-04-15T15:23:55.271660+0000","last_unstale":"2024-04-15T15:23:56.416048+0000","last_undegraded":"2024-04-15T15:23:56.416048+0000","last_fullsized":"2024-04-15T15:23:56.416048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419745+0000","last_change":"2024-04-15T15:23:54.251693+0000","last_active":"2024-04-15T15:23:56.419745+0000","last_peered":"2024-04-15T15:23:56.419745+0000","last_clean":"2024-04-15T15:23:56.419745+0000","last_became_active":"2024-04-15T15:23:54.250255+0000","last_became_peered":"2024-04-15T15:23:54.250255+0000","last_unstale":"2024-04-15T15:23:56.419745+0000","last_undegraded":"2024-04-15T15:23:56.419745+0000","last_fullsized":"2024-04-15T15:23:56.419745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:08:28.726161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261853+0000","last_change":"2024-04-15T15:23:55.278856+0000","last_active":"2024-04-15T15:23:56.261853+0000","last_peered":"2024-04-15T15:23:56.261853+0000","last_clean":"2024-04-15T15:23:56.261853+0000","last_became_active":"2024-04-15T15:23:55.278605+0000","last_became_peered":"2024-04-15T15:23:55.278605+0000","last_unstale":"2024-04-15T15:23:56.261853+0000","last_undegraded":"2024-04-15T15:23:56.261853+0000","last_fullsized":"2024-04-15T15:23:56.261853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.264390+0000","last_change":"2024-04-15T15:23:55.260285+0000","last_active":"2024-04-15T15:23:56.264390+0000","last_peered":"2024-04-15T15:23:56.264390+0000","last_clean":"2024-04-15T15:23:56.264390+0000","last_became_active":"2024-04-15T15:23:55.260016+0000","last_became_peered":"2024-04-15T15:23:55.260016+0000","last_unstale":"2024-04-15T15:23:56.264390+0000","last_undegraded":"2024-04-15T15:23:56.264390+0000","last_fullsized":"2024-04-15T15:23:56.264390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:58.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418779+0000","last_change":"2024-04-15T15:23:54.250621+0000","last_active":"2024-04-15T15:23:56.418779+0000","last_peered":"2024-04-15T15:23:56.418779+0000","last_clean":"2024-04-15T15:23:56.418779+0000","last_became_active":"2024-04-15T15:23:54.250204+0000","last_became_peered":"2024-04-15T15:23:54.250204+0000","last_unstale":"2024-04-15T15:23:56.418779+0000","last_undegraded":"2024-04-15T15:23:56.418779+0000","last_fullsized":"2024-04-15T15:23:56.418779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:54:51.507224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262544+0000","last_change":"2024-04-15T15:23:55.263054+0000","last_active":"2024-04-15T15:23:56.262544+0000","last_peered":"2024-04-15T15:23:56.262544+0000","last_clean":"2024-04-15T15:23:56.262544+0000","last_became_active":"2024-04-15T15:23:55.262742+0000","last_became_peered":"2024-04-15T15:23:55.262742+0000","last_unstale":"2024-04-15T15:23:56.262544+0000","last_undegraded":"2024-04-15T15:23:56.262544+0000","last_fullsized":"2024-04-15T15:23:56.262544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:30:18.931762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418457+0000","last_change":"2024-04-15T15:23:54.250572+0000","last_active":"2024-04-15T15:23:56.418457+0000","last_peered":"2024-04-15T15:23:56.418457+0000","last_clean":"2024-04-15T15:23:56.418457+0000","last_became_active":"2024-04-15T15:23:54.250229+0000","last_became_peered":"2024-04-15T15:23:54.250229+0000","last_unstale":"2024-04-15T15:23:56.418457+0000","last_undegraded":"2024-04-15T15:23:56.418457+0000","last_fullsized":"2024-04-15T15:23:56.418457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:15.052794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416985+0000","last_change":"2024-04-15T15:23:55.269263+0000","last_active":"2024-04-15T15:23:56.416985+0000","last_peered":"2024-04-15T15:23:56.416985+0000","last_clean":"2024-04-15T15:23:56.416985+0000","last_became_active":"2024-04-15T15:23:55.268706+0000","last_became_peered":"2024-04-15T15:23:55.268706+0000","last_unstale":"2024-04-15T15:23:56.416985+0000","last_undegraded":"2024-04-15T15:23:56.416985+0000","last_fullsized":"2024-04-15T15:23:56.416985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.057703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415538+0000","last_change":"2024-04-15T15:23:54.229794+0000","last_active":"2024-04-15T15:23:56.415538+0000","last_peered":"2024-04-15T15:23:56.415538+0000","last_clean":"2024-04-15T15:23:56.415538+0000","last_became_active":"2024-04-15T15:23:54.229509+0000","last_became_peered":"2024-04-15T15:23:54.229509+0000","last_unstale":"2024-04-15T15:23:56.415538+0000","last_undegraded":"2024-04-15T15:23:56.415538+0000","last_fullsized":"2024-04-15T15:23:56.415538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:49:04.203638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415289+0000","last_change":"2024-04-15T15:23:55.274587+0000","last_active":"2024-04-15T15:23:56.415289+0000","last_peered":"2024-04-15T15:23:56.415289+0000","last_clean":"2024-04-15T15:23:56.415289+0000","last_became_active":"2024-04-15T15:23:55.274127+0000","last_became_peered":"2024-04-15T15:23:55.274127+0000","last_unstale":"2024-04-15T15:23:56.415289+0000","last_undegraded":"2024-04-15T15:23:56.415289+0000","last_fullsized":"2024-04-15T15:23:56.415289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417220+0000","last_change":"2024-04-15T15:23:54.236884+0000","last_active":"2024-04-15T15:23:56.417220+0000","last_peered":"2024-04-15T15:23:56.417220+0000","last_clean":"2024-04-15T15:23:56.417220+0000","last_became_active":"2024-04-15T15:23:54.236583+0000","last_became_peered":"2024-04-15T15:23:54.236583+0000","last_unstale":"2024-04-15T15:23:56.417220+0000","last_undegraded":"2024-04-15T15:23:56.417220+0000","last_fullsized":"2024-04-15T15:23:56.417220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:24:05.302 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:24:05.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.546+0000 7efc6ec00640 1 Processor -- start 2024-04-15T15:24:05.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.546+0000 7efc6ec00640 1 -- start start 2024-04-15T15:24:05.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 0x7efc680cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6ec00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efc680cdf50 0x7efc6809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6ec00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc680d12b0 con 0x7efc6815fe50 2024-04-15T15:24:05.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6ec00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7efc680d0670 con 0x7efc680cdf50 2024-04-15T15:24:05.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc68071e40 con 0x7efc680d0190 2024-04-15T15:24:05.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6d800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7efc6815fe50 0x7efc680d0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33548/0 (socket says 172.21.15.150:33548) 2024-04-15T15:24:05.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6e200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efc680cdf50 0x7efc6809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6d800640 1 -- 172.21.15.150:0/1376451683 learned_addr learned my addr 172.21.15.150:0/1376451683 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:05.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.548+0000 7efc6ce00640 1 --2- 172.21.15.150:0/1376451683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 0x7efc680cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.549+0000 7efc6e200640 1 -- 172.21.15.150:0/1376451683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 msgr2=0x7efc680cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:05.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.549+0000 7efc6e200640 1 --2- 172.21.15.150:0/1376451683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 0x7efc680cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.549+0000 7efc6e200640 1 -- 172.21.15.150:0/1376451683 >> v1:172.21.15.150:6789/0 conn(0x7efc6815fe50 legacy=0x7efc680d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.549+0000 7efc6e200640 1 -- 172.21.15.150:0/1376451683 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc68072120 con 0x7efc680cdf50 2024-04-15T15:24:05.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.549+0000 7efc67e00640 1 -- 172.21.15.150:0/1376451683 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888880960 0 0) 0x7efc5c001110 con 0x7efc6815fe50 2024-04-15T15:24:05.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.550+0000 7efc6e200640 1 --2- 172.21.15.150:0/1376451683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc680cdf50 0x7efc6809aa40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efc600097a0 tx=0x7efc6002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45f997a8f2fd9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:05.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.550+0000 7efc67e00640 1 -- 172.21.15.150:0/1376451683 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc600315a0 con 0x7efc680cdf50 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.550+0000 7efc67e00640 1 -- 172.21.15.150:0/1376451683 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc60037070 con 0x7efc680cdf50 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.550+0000 7efc67e00640 1 -- 172.21.15.150:0/1376451683 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60031960 con 0x7efc680cdf50 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.551+0000 7efc6ec00640 1 -- 172.21.15.150:0/1376451683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc680cdf50 msgr2=0x7efc6809aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.551+0000 7efc6ec00640 1 --2- 172.21.15.150:0/1376451683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc680cdf50 0x7efc6809aa40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efc600097a0 tx=0x7efc6002b840 comp rx=0 tx=0).stop 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.551+0000 7efc6ec00640 1 -- 172.21.15.150:0/1376451683 shutdown_connections 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.551+0000 7efc6ec00640 1 --2- 172.21.15.150:0/1376451683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 0x7efc680cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.551+0000 7efc6ec00640 1 --2- 172.21.15.150:0/1376451683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc680cdf50 0x7efc6809aa40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.551+0000 7efc6ec00640 1 -- 172.21.15.150:0/1376451683 >> 172.21.15.150:0/1376451683 conn(0x7efc680d25c0 msgr2=0x7efc680d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.551+0000 7efc6ec00640 1 -- 172.21.15.150:0/1376451683 shutdown_connections 2024-04-15T15:24:05.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.551+0000 7efc6ec00640 1 -- 172.21.15.150:0/1376451683 wait complete. 2024-04-15T15:24:05.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.552+0000 7efc6ec00640 1 Processor -- start 2024-04-15T15:24:05.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.553+0000 7efc6ec00640 1 -- start start 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.553+0000 7efc6ec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc680cdf50 0x7efc68157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.553+0000 7efc6ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 0x7efc680701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.553+0000 7efc6ec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc6815fe50 0x7efc68070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.553+0000 7efc6ec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc680cedc0 con 0x7efc6815fe50 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.553+0000 7efc6ec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc680cec40 con 0x7efc680cdf50 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.553+0000 7efc6ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc6809af80 con 0x7efc680d0190 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc6815fe50 0x7efc68070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc6815fe50 0x7efc68070730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44800/0 (socket says 172.21.15.150:44800) 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 -- 172.21.15.150:0/4000885565 learned_addr learned my addr 172.21.15.150:0/4000885565 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6ce00640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 0x7efc680701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 -- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 msgr2=0x7efc680701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 0x7efc680701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 -- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc680cdf50 msgr2=0x7efc68157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc680cdf50 0x7efc68157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 -- 172.21.15.150:0/4000885565 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc681e7250 con 0x7efc6815fe50 2024-04-15T15:24:05.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.554+0000 7efc6e200640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc6815fe50 0x7efc68070730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc60034040 tx=0x7efc600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:05.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.555+0000 7efc65600640 1 -- 172.21.15.150:0/4000885565 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60032630 con 0x7efc6815fe50 2024-04-15T15:24:05.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.555+0000 7efc65600640 1 -- 172.21.15.150:0/4000885565 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc600327d0 con 0x7efc6815fe50 2024-04-15T15:24:05.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.555+0000 7efc65600640 1 -- 172.21.15.150:0/4000885565 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60032bc0 con 0x7efc6815fe50 2024-04-15T15:24:05.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.555+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc681e7530 con 0x7efc6815fe50 2024-04-15T15:24:05.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.555+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc681e64b0 con 0x7efc6815fe50 2024-04-15T15:24:05.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.556+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc680755c0 con 0x7efc6815fe50 2024-04-15T15:24:05.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.557+0000 7efc65600640 1 -- 172.21.15.150:0/4000885565 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc6004c020 con 0x7efc6815fe50 2024-04-15T15:24:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.558+0000 7efc65600640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc4c06c8f0 0x7efc4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.558+0000 7efc65600640 1 -- 172.21.15.150:0/4000885565 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efc6002f020 con 0x7efc6815fe50 2024-04-15T15:24:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.559+0000 7efc6d800640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc4c06c8f0 0x7efc4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:05.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.560+0000 7efc6d800640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc4c06c8f0 0x7efc4c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc5c007fd0 tx=0x7efc5c007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:05.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.562+0000 7efc65600640 1 -- 172.21.15.150:0/4000885565 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc60040070 con 0x7efc6815fe50 2024-04-15T15:24:05.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.744+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc680b8540 con 0x7efc4c06c8f0 2024-04-15T15:24:05.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.762+0000 7efc65600640 1 -- 172.21.15.150:0/4000885565 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7efc680b8540 con 0x7efc4c06c8f0 2024-04-15T15:24:05.767 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:05.768 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.767+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc4c06c8f0 msgr2=0x7efc4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.767+0000 7efc6ec00640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc4c06c8f0 0x7efc4c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc5c007fd0 tx=0x7efc5c007960 comp rx=0 tx=0).stop 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.767+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc6815fe50 msgr2=0x7efc68070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.767+0000 7efc6ec00640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc6815fe50 0x7efc68070730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc60034040 tx=0x7efc600046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.768+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 shutdown_connections 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.768+0000 7efc6ec00640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc6815fe50 0x7efc68070730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.768+0000 7efc6ec00640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc4c06c8f0 0x7efc4c06edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.768+0000 7efc6ec00640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc680d0190 0x7efc680701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.768+0000 7efc6ec00640 1 --2- 172.21.15.150:0/4000885565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc680cdf50 0x7efc68157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.768+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 >> 172.21.15.150:0/4000885565 conn(0x7efc680d25c0 msgr2=0x7efc680d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.768+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 shutdown_connections 2024-04-15T15:24:05.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:05.768+0000 7efc6ec00640 1 -- 172.21.15.150:0/4000885565 wait complete. 2024-04-15T15:24:05.788 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-15T15:24:05.219982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.992919"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418140+0000","last_change":"2024-04-15T15:23:55.263891+0000","last_active":"2024-04-15T15:23:56.418140+0000","last_peered":"2024-04-15T15:23:56.418140+0000","last_clean":"2024-04-15T15:23:56.418140+0000","last_became_active":"2024-04-15T15:23:55.263577+0000","last_became_peered":"2024-04-15T15:23:55.263577+0000","last_unstale":"2024-04-15T15:23:56.418140+0000","last_undegraded":"2024-04-15T15:23:56.418140+0000","last_fullsized":"2024-04-15T15:23:56.418140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:58.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255982+0000","last_change":"2024-04-15T15:23:55.259842+0000","last_active":"2024-04-15T15:23:56.255982+0000","last_peered":"2024-04-15T15:23:56.255982+0000","last_clean":"2024-04-15T15:23:56.255982+0000","last_became_active":"2024-04-15T15:23:55.259375+0000","last_became_peered":"2024-04-15T15:23:55.259375+0000","last_unstale":"2024-04-15T15:23:56.255982+0000","last_undegraded":"2024-04-15T15:23:56.255982+0000","last_fullsized":"2024-04-15T15:23:56.255982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418953+0000","last_change":"2024-04-15T15:23:55.249464+0000","last_active":"2024-04-15T15:23:56.418953+0000","last_peered":"2024-04-15T15:23:56.418953+0000","last_clean":"2024-04-15T15:23:56.418953+0000","last_became_active":"2024-04-15T15:23:55.248697+0000","last_became_peered":"2024-04-15T15:23:55.248697+0000","last_unstale":"2024-04-15T15:23:56.418953+0000","last_undegraded":"2024-04-15T15:23:56.418953+0000","last_fullsized":"2024-04-15T15:23:56.418953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262156+0000","last_change":"2024-04-15T15:23:55.267985+0000","last_active":"2024-04-15T15:23:56.262156+0000","last_peered":"2024-04-15T15:23:56.262156+0000","last_clean":"2024-04-15T15:23:56.262156+0000","last_became_active":"2024-04-15T15:23:55.267575+0000","last_became_peered":"2024-04-15T15:23:55.267575+0000","last_unstale":"2024-04-15T15:23:56.262156+0000","last_undegraded":"2024-04-15T15:23:56.262156+0000","last_fullsized":"2024-04-15T15:23:56.262156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:11.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419665+0000","last_change":"2024-04-15T15:23:55.260235+0000","last_active":"2024-04-15T15:23:56.419665+0000","last_peered":"2024-04-15T15:23:56.419665+0000","last_clean":"2024-04-15T15:23:56.419665+0000","last_became_active":"2024-04-15T15:23:55.259947+0000","last_became_peered":"2024-04-15T15:23:55.259947+0000","last_unstale":"2024-04-15T15:23:56.419665+0000","last_undegraded":"2024-04-15T15:23:56.419665+0000","last_fullsized":"2024-04-15T15:23:56.419665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:18:41.223608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261963+0000","last_change":"2024-04-15T15:23:55.257936+0000","last_active":"2024-04-15T15:23:56.261963+0000","last_peered":"2024-04-15T15:23:56.261963+0000","last_clean":"2024-04-15T15:23:56.261963+0000","last_became_active":"2024-04-15T15:23:55.257397+0000","last_became_peered":"2024-04-15T15:23:55.257397+0000","last_unstale":"2024-04-15T15:23:56.261963+0000","last_undegraded":"2024-04-15T15:23:56.261963+0000","last_fullsized":"2024-04-15T15:23:56.261963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417763+0000","last_change":"2024-04-15T15:23:55.265207+0000","last_active":"2024-04-15T15:23:56.417763+0000","last_peered":"2024-04-15T15:23:56.417763+0000","last_clean":"2024-04-15T15:23:56.417763+0000","last_became_active":"2024-04-15T15:23:55.264857+0000","last_became_peered":"2024-04-15T15:23:55.264857+0000","last_unstale":"2024-04-15T15:23:56.417763+0000","last_undegraded":"2024-04-15T15:23:56.417763+0000","last_fullsized":"2024-04-15T15:23:56.417763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:35.342060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416683+0000","last_change":"2024-04-15T15:23:55.270590+0000","last_active":"2024-04-15T15:23:56.416683+0000","last_peered":"2024-04-15T15:23:56.416683+0000","last_clean":"2024-04-15T15:23:56.416683+0000","last_became_active":"2024-04-15T15:23:55.267976+0000","last_became_peered":"2024-04-15T15:23:55.267976+0000","last_unstale":"2024-04-15T15:23:56.416683+0000","last_undegraded":"2024-04-15T15:23:56.416683+0000","last_fullsized":"2024-04-15T15:23:56.416683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:50:11.769633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418517+0000","last_change":"2024-04-15T15:23:55.274056+0000","last_active":"2024-04-15T15:23:56.418517+0000","last_peered":"2024-04-15T15:23:56.418517+0000","last_clean":"2024-04-15T15:23:56.418517+0000","last_became_active":"2024-04-15T15:23:55.273746+0000","last_became_peered":"2024-04-15T15:23:55.273746+0000","last_unstale":"2024-04-15T15:23:56.418517+0000","last_undegraded":"2024-04-15T15:23:56.418517+0000","last_fullsized":"2024-04-15T15:23:56.418517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:31:11.922705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262425+0000","last_change":"2024-04-15T15:23:55.272281+0000","last_active":"2024-04-15T15:23:56.262425+0000","last_peered":"2024-04-15T15:23:56.262425+0000","last_clean":"2024-04-15T15:23:56.262425+0000","last_became_active":"2024-04-15T15:23:55.269795+0000","last_became_peered":"2024-04-15T15:23:55.269795+0000","last_unstale":"2024-04-15T15:23:56.262425+0000","last_undegraded":"2024-04-15T15:23:56.262425+0000","last_fullsized":"2024-04-15T15:23:56.262425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263663+0000","last_change":"2024-04-15T15:23:55.269199+0000","last_active":"2024-04-15T15:23:56.263663+0000","last_peered":"2024-04-15T15:23:56.263663+0000","last_clean":"2024-04-15T15:23:56.263663+0000","last_became_active":"2024-04-15T15:23:55.268870+0000","last_became_peered":"2024-04-15T15:23:55.268870+0000","last_unstale":"2024-04-15T15:23:56.263663+0000","last_undegraded":"2024-04-15T15:23:56.263663+0000","last_fullsized":"2024-04-15T15:23:56.263663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418307+0000","last_change":"2024-04-15T15:23:55.263524+0000","last_active":"2024-04-15T15:23:56.418307+0000","last_peered":"2024-04-15T15:23:56.418307+0000","last_clean":"2024-04-15T15:23:56.418307+0000","last_became_active":"2024-04-15T15:23:55.263210+0000","last_became_peered":"2024-04-15T15:23:55.263210+0000","last_unstale":"2024-04-15T15:23:56.418307+0000","last_undegraded":"2024-04-15T15:23:56.418307+0000","last_fullsized":"2024-04-15T15:23:56.418307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:32.154491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415704+0000","last_change":"2024-04-15T15:23:55.272996+0000","last_active":"2024-04-15T15:23:56.415704+0000","last_peered":"2024-04-15T15:23:56.415704+0000","last_clean":"2024-04-15T15:23:56.415704+0000","last_became_active":"2024-04-15T15:23:55.272701+0000","last_became_peered":"2024-04-15T15:23:55.272701+0000","last_unstale":"2024-04-15T15:23:56.415704+0000","last_undegraded":"2024-04-15T15:23:56.415704+0000","last_fullsized":"2024-04-15T15:23:56.415704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417462+0000","last_change":"2024-04-15T15:23:55.260534+0000","last_active":"2024-04-15T15:23:56.417462+0000","last_peered":"2024-04-15T15:23:56.417462+0000","last_clean":"2024-04-15T15:23:56.417462+0000","last_became_active":"2024-04-15T15:23:55.258506+0000","last_became_peered":"2024-04-15T15:23:55.258506+0000","last_unstale":"2024-04-15T15:23:56.417462+0000","last_undegraded":"2024-04-15T15:23:56.417462+0000","last_fullsized":"2024-04-15T15:23:56.417462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255395+0000","last_change":"2024-04-15T15:23:55.265936+0000","last_active":"2024-04-15T15:23:56.255395+0000","last_peered":"2024-04-15T15:23:56.255395+0000","last_clean":"2024-04-15T15:23:56.255395+0000","last_became_active":"2024-04-15T15:23:55.265118+0000","last_became_peered":"2024-04-15T15:23:55.265118+0000","last_unstale":"2024-04-15T15:23:56.255395+0000","last_undegraded":"2024-04-15T15:23:56.255395+0000","last_fullsized":"2024-04-15T15:23:56.255395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.260985+0000","last_change":"2024-04-15T15:23:55.252289+0000","last_active":"2024-04-15T15:23:56.260985+0000","last_peered":"2024-04-15T15:23:56.260985+0000","last_clean":"2024-04-15T15:23:56.260985+0000","last_became_active":"2024-04-15T15:23:55.252007+0000","last_became_peered":"2024-04-15T15:23:55.252007+0000","last_unstale":"2024-04-15T15:23:56.260985+0000","last_undegraded":"2024-04-15T15:23:56.260985+0000","last_fullsized":"2024-04-15T15:23:56.260985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262956+0000","last_change":"2024-04-15T15:23:55.256170+0000","last_active":"2024-04-15T15:23:56.262956+0000","last_peered":"2024-04-15T15:23:56.262956+0000","last_clean":"2024-04-15T15:23:56.262956+0000","last_became_active":"2024-04-15T15:23:55.255896+0000","last_became_peered":"2024-04-15T15:23:55.255896+0000","last_unstale":"2024-04-15T15:23:56.262956+0000","last_undegraded":"2024-04-15T15:23:56.262956+0000","last_fullsized":"2024-04-15T15:23:56.262956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:16:50.418929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417610+0000","last_change":"2024-04-15T15:23:55.269766+0000","last_active":"2024-04-15T15:23:56.417610+0000","last_peered":"2024-04-15T15:23:56.417610+0000","last_clean":"2024-04-15T15:23:56.417610+0000","last_became_active":"2024-04-15T15:23:55.269443+0000","last_became_peered":"2024-04-15T15:23:55.269443+0000","last_unstale":"2024-04-15T15:23:56.417610+0000","last_undegraded":"2024-04-15T15:23:56.417610+0000","last_fullsized":"2024-04-15T15:23:56.417610+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:43:55.932310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419019+0000","last_change":"2024-04-15T15:23:55.265016+0000","last_active":"2024-04-15T15:23:56.419019+0000","last_peered":"2024-04-15T15:23:56.419019+0000","last_clean":"2024-04-15T15:23:56.419019+0000","last_became_active":"2024-04-15T15:23:55.258660+0000","last_became_peered":"2024-04-15T15:23:55.258660+0000","last_unstale":"2024-04-15T15:23:56.419019+0000","last_undegraded":"2024-04-15T15:23:56.419019+0000","last_fullsized":"2024-04-15T15:23:56.419019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:08:08.360698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262025+0000","last_change":"2024-04-15T15:23:55.275816+0000","last_active":"2024-04-15T15:23:56.262025+0000","last_peered":"2024-04-15T15:23:56.262025+0000","last_clean":"2024-04-15T15:23:56.262025+0000","last_became_active":"2024-04-15T15:23:55.274960+0000","last_became_peered":"2024-04-15T15:23:55.274960+0000","last_unstale":"2024-04-15T15:23:56.262025+0000","last_undegraded":"2024-04-15T15:23:56.262025+0000","last_fullsized":"2024-04-15T15:23:56.262025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417084+0000","last_change":"2024-04-15T15:23:55.267469+0000","last_active":"2024-04-15T15:23:56.417084+0000","last_peered":"2024-04-15T15:23:56.417084+0000","last_clean":"2024-04-15T15:23:56.417084+0000","last_became_active":"2024-04-15T15:23:55.266878+0000","last_became_peered":"2024-04-15T15:23:55.266878+0000","last_unstale":"2024-04-15T15:23:56.417084+0000","last_undegraded":"2024-04-15T15:23:56.417084+0000","last_fullsized":"2024-04-15T15:23:56.417084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262363+0000","last_change":"2024-04-15T15:23:55.262231+0000","last_active":"2024-04-15T15:23:56.262363+0000","last_peered":"2024-04-15T15:23:56.262363+0000","last_clean":"2024-04-15T15:23:56.262363+0000","last_became_active":"2024-04-15T15:23:55.261861+0000","last_became_peered":"2024-04-15T15:23:55.261861+0000","last_unstale":"2024-04-15T15:23:56.262363+0000","last_undegraded":"2024-04-15T15:23:56.262363+0000","last_fullsized":"2024-04-15T15:23:56.262363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:38.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417901+0000","last_change":"2024-04-15T15:23:55.275456+0000","last_active":"2024-04-15T15:23:56.417901+0000","last_peered":"2024-04-15T15:23:56.417901+0000","last_clean":"2024-04-15T15:23:56.417901+0000","last_became_active":"2024-04-15T15:23:55.275108+0000","last_became_peered":"2024-04-15T15:23:55.275108+0000","last_unstale":"2024-04-15T15:23:56.417901+0000","last_undegraded":"2024-04-15T15:23:56.417901+0000","last_fullsized":"2024-04-15T15:23:56.417901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:38.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261274+0000","last_change":"2024-04-15T15:23:55.270445+0000","last_active":"2024-04-15T15:23:56.261274+0000","last_peered":"2024-04-15T15:23:56.261274+0000","last_clean":"2024-04-15T15:23:56.261274+0000","last_became_active":"2024-04-15T15:23:55.270131+0000","last_became_peered":"2024-04-15T15:23:55.270131+0000","last_unstale":"2024-04-15T15:23:56.261274+0000","last_undegraded":"2024-04-15T15:23:56.261274+0000","last_fullsized":"2024-04-15T15:23:56.261274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415461+0000","last_change":"2024-04-15T15:23:55.260659+0000","last_active":"2024-04-15T15:23:56.415461+0000","last_peered":"2024-04-15T15:23:56.415461+0000","last_clean":"2024-04-15T15:23:56.415461+0000","last_became_active":"2024-04-15T15:23:55.258739+0000","last_became_peered":"2024-04-15T15:23:55.258739+0000","last_unstale":"2024-04-15T15:23:56.415461+0000","last_undegraded":"2024-04-15T15:23:56.415461+0000","last_fullsized":"2024-04-15T15:23:56.415461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262634+0000","last_change":"2024-04-15T15:23:55.272818+0000","last_active":"2024-04-15T15:23:56.262634+0000","last_peered":"2024-04-15T15:23:56.262634+0000","last_clean":"2024-04-15T15:23:56.262634+0000","last_became_active":"2024-04-15T15:23:55.269858+0000","last_became_peered":"2024-04-15T15:23:55.269858+0000","last_unstale":"2024-04-15T15:23:56.262634+0000","last_undegraded":"2024-04-15T15:23:56.262634+0000","last_fullsized":"2024-04-15T15:23:56.262634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255726+0000","last_change":"2024-04-15T15:23:55.261128+0000","last_active":"2024-04-15T15:23:56.255726+0000","last_peered":"2024-04-15T15:23:56.255726+0000","last_clean":"2024-04-15T15:23:56.255726+0000","last_became_active":"2024-04-15T15:23:55.259248+0000","last_became_peered":"2024-04-15T15:23:55.259248+0000","last_unstale":"2024-04-15T15:23:56.255726+0000","last_undegraded":"2024-04-15T15:23:56.255726+0000","last_fullsized":"2024-04-15T15:23:56.255726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415563+0000","last_change":"2024-04-15T15:23:55.264402+0000","last_active":"2024-04-15T15:23:56.415563+0000","last_peered":"2024-04-15T15:23:56.415563+0000","last_clean":"2024-04-15T15:23:56.415563+0000","last_became_active":"2024-04-15T15:23:55.264083+0000","last_became_peered":"2024-04-15T15:23:55.264083+0000","last_unstale":"2024-04-15T15:23:56.415563+0000","last_undegraded":"2024-04-15T15:23:56.415563+0000","last_fullsized":"2024-04-15T15:23:56.415563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417369+0000","last_change":"2024-04-15T15:23:55.262147+0000","last_active":"2024-04-15T15:23:56.417369+0000","last_peered":"2024-04-15T15:23:56.417369+0000","last_clean":"2024-04-15T15:23:56.417369+0000","last_became_active":"2024-04-15T15:23:55.261524+0000","last_became_peered":"2024-04-15T15:23:55.261524+0000","last_unstale":"2024-04-15T15:23:56.417369+0000","last_undegraded":"2024-04-15T15:23:56.417369+0000","last_fullsized":"2024-04-15T15:23:56.417369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262811+0000","last_change":"2024-04-15T15:23:55.267385+0000","last_active":"2024-04-15T15:23:56.262811+0000","last_peered":"2024-04-15T15:23:56.262811+0000","last_clean":"2024-04-15T15:23:56.262811+0000","last_became_active":"2024-04-15T15:23:55.267045+0000","last_became_peered":"2024-04-15T15:23:55.267045+0000","last_unstale":"2024-04-15T15:23:56.262811+0000","last_undegraded":"2024-04-15T15:23:56.262811+0000","last_fullsized":"2024-04-15T15:23:56.262811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:28:35.342060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417143+0000","last_change":"2024-04-15T15:23:55.261039+0000","last_active":"2024-04-15T15:23:56.417143+0000","last_peered":"2024-04-15T15:23:56.417143+0000","last_clean":"2024-04-15T15:23:56.417143+0000","last_became_active":"2024-04-15T15:23:55.260620+0000","last_became_peered":"2024-04-15T15:23:55.260620+0000","last_unstale":"2024-04-15T15:23:56.417143+0000","last_undegraded":"2024-04-15T15:23:56.417143+0000","last_fullsized":"2024-04-15T15:23:56.417143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:27:44.245014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261366+0000","last_change":"2024-04-15T15:23:55.271261+0000","last_active":"2024-04-15T15:23:56.261366+0000","last_peered":"2024-04-15T15:23:56.261366+0000","last_clean":"2024-04-15T15:23:56.261366+0000","last_became_active":"2024-04-15T15:23:55.270954+0000","last_became_peered":"2024-04-15T15:23:55.270954+0000","last_unstale":"2024-04-15T15:23:56.261366+0000","last_undegraded":"2024-04-15T15:23:56.261366+0000","last_fullsized":"2024-04-15T15:23:56.261366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417469+0000","last_change":"2024-04-15T15:23:55.263715+0000","last_active":"2024-04-15T15:23:56.417469+0000","last_peered":"2024-04-15T15:23:56.417469+0000","last_clean":"2024-04-15T15:23:56.417469+0000","last_became_active":"2024-04-15T15:23:55.256992+0000","last_became_peered":"2024-04-15T15:23:55.256992+0000","last_unstale":"2024-04-15T15:23:56.417469+0000","last_undegraded":"2024-04-15T15:23:56.417469+0000","last_fullsized":"2024-04-15T15:23:56.417469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:25.936170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416848+0000","last_change":"2024-04-15T15:23:54.250784+0000","last_active":"2024-04-15T15:23:56.416848+0000","last_peered":"2024-04-15T15:23:56.416848+0000","last_clean":"2024-04-15T15:23:56.416848+0000","last_became_active":"2024-04-15T15:23:54.250347+0000","last_became_peered":"2024-04-15T15:23:54.250347+0000","last_unstale":"2024-04-15T15:23:56.416848+0000","last_undegraded":"2024-04-15T15:23:56.416848+0000","last_fullsized":"2024-04-15T15:23:56.416848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417629+0000","last_change":"2024-04-15T15:23:55.262408+0000","last_active":"2024-04-15T15:23:56.417629+0000","last_peered":"2024-04-15T15:23:56.417629+0000","last_clean":"2024-04-15T15:23:56.417629+0000","last_became_active":"2024-04-15T15:23:55.261815+0000","last_became_peered":"2024-04-15T15:23:55.261815+0000","last_unstale":"2024-04-15T15:23:56.417629+0000","last_undegraded":"2024-04-15T15:23:56.417629+0000","last_fullsized":"2024-04-15T15:23:56.417629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:30:45.151436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261683+0000","last_change":"2024-04-15T15:23:54.239676+0000","last_active":"2024-04-15T15:23:56.261683+0000","last_peered":"2024-04-15T15:23:56.261683+0000","last_clean":"2024-04-15T15:23:56.261683+0000","last_became_active":"2024-04-15T15:23:54.239084+0000","last_became_peered":"2024-04-15T15:23:54.239084+0000","last_unstale":"2024-04-15T15:23:56.261683+0000","last_undegraded":"2024-04-15T15:23:56.261683+0000","last_fullsized":"2024-04-15T15:23:56.261683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415034+0000","last_change":"2024-04-15T15:23:54.241378+0000","last_active":"2024-04-15T15:23:56.415034+0000","last_peered":"2024-04-15T15:23:56.415034+0000","last_clean":"2024-04-15T15:23:56.415034+0000","last_became_active":"2024-04-15T15:23:54.241084+0000","last_became_peered":"2024-04-15T15:23:54.241084+0000","last_unstale":"2024-04-15T15:23:56.415034+0000","last_undegraded":"2024-04-15T15:23:56.415034+0000","last_fullsized":"2024-04-15T15:23:56.415034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:38:14.692145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.414885+0000","last_change":"2024-04-15T15:23:55.257791+0000","last_active":"2024-04-15T15:23:56.414885+0000","last_peered":"2024-04-15T15:23:56.414885+0000","last_clean":"2024-04-15T15:23:56.414885+0000","last_became_active":"2024-04-15T15:23:55.257531+0000","last_became_peered":"2024-04-15T15:23:55.257531+0000","last_unstale":"2024-04-15T15:23:56.414885+0000","last_undegraded":"2024-04-15T15:23:56.414885+0000","last_fullsized":"2024-04-15T15:23:56.414885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:26:41.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261118+0000","last_change":"2024-04-15T15:23:54.235199+0000","last_active":"2024-04-15T15:23:56.261118+0000","last_peered":"2024-04-15T15:23:56.261118+0000","last_clean":"2024-04-15T15:23:56.261118+0000","last_became_active":"2024-04-15T15:23:54.233620+0000","last_became_peered":"2024-04-15T15:23:54.233620+0000","last_unstale":"2024-04-15T15:23:56.261118+0000","last_undegraded":"2024-04-15T15:23:56.261118+0000","last_fullsized":"2024-04-15T15:23:56.261118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255313+0000","last_change":"2024-04-15T15:23:55.261632+0000","last_active":"2024-04-15T15:23:56.255313+0000","last_peered":"2024-04-15T15:23:56.255313+0000","last_clean":"2024-04-15T15:23:56.255313+0000","last_became_active":"2024-04-15T15:23:55.261351+0000","last_became_peered":"2024-04-15T15:23:55.261351+0000","last_unstale":"2024-04-15T15:23:56.255313+0000","last_undegraded":"2024-04-15T15:23:56.255313+0000","last_fullsized":"2024-04-15T15:23:56.255313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263032+0000","last_change":"2024-04-15T15:23:54.237429+0000","last_active":"2024-04-15T15:23:56.263032+0000","last_peered":"2024-04-15T15:23:56.263032+0000","last_clean":"2024-04-15T15:23:56.263032+0000","last_became_active":"2024-04-15T15:23:54.237045+0000","last_became_peered":"2024-04-15T15:23:54.237045+0000","last_unstale":"2024-04-15T15:23:56.263032+0000","last_undegraded":"2024-04-15T15:23:56.263032+0000","last_fullsized":"2024-04-15T15:23:56.263032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262636+0000","last_change":"2024-04-15T15:23:55.263292+0000","last_active":"2024-04-15T15:23:56.262636+0000","last_peered":"2024-04-15T15:23:56.262636+0000","last_clean":"2024-04-15T15:23:56.262636+0000","last_became_active":"2024-04-15T15:23:55.261899+0000","last_became_peered":"2024-04-15T15:23:55.261899+0000","last_unstale":"2024-04-15T15:23:56.262636+0000","last_undegraded":"2024-04-15T15:23:56.262636+0000","last_fullsized":"2024-04-15T15:23:56.262636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:18:40.716704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262724+0000","last_change":"2024-04-15T15:23:54.242528+0000","last_active":"2024-04-15T15:23:56.262724+0000","last_peered":"2024-04-15T15:23:56.262724+0000","last_clean":"2024-04-15T15:23:56.262724+0000","last_became_active":"2024-04-15T15:23:54.242233+0000","last_became_peered":"2024-04-15T15:23:54.242233+0000","last_unstale":"2024-04-15T15:23:56.262724+0000","last_undegraded":"2024-04-15T15:23:56.262724+0000","last_fullsized":"2024-04-15T15:23:56.262724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417168+0000","last_change":"2024-04-15T15:23:55.268964+0000","last_active":"2024-04-15T15:23:56.417168+0000","last_peered":"2024-04-15T15:23:56.417168+0000","last_clean":"2024-04-15T15:23:56.417168+0000","last_became_active":"2024-04-15T15:23:55.268348+0000","last_became_peered":"2024-04-15T15:23:55.268348+0000","last_unstale":"2024-04-15T15:23:56.417168+0000","last_undegraded":"2024-04-15T15:23:56.417168+0000","last_fullsized":"2024-04-15T15:23:56.417168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255623+0000","last_change":"2024-04-15T15:23:54.238578+0000","last_active":"2024-04-15T15:23:56.255623+0000","last_peered":"2024-04-15T15:23:56.255623+0000","last_clean":"2024-04-15T15:23:56.255623+0000","last_became_active":"2024-04-15T15:23:54.238332+0000","last_became_peered":"2024-04-15T15:23:54.238332+0000","last_unstale":"2024-04-15T15:23:56.255623+0000","last_undegraded":"2024-04-15T15:23:56.255623+0000","last_fullsized":"2024-04-15T15:23:56.255623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418641+0000","last_change":"2024-04-15T15:23:55.266173+0000","last_active":"2024-04-15T15:23:56.418641+0000","last_peered":"2024-04-15T15:23:56.418641+0000","last_clean":"2024-04-15T15:23:56.418641+0000","last_became_active":"2024-04-15T15:23:55.265877+0000","last_became_peered":"2024-04-15T15:23:55.265877+0000","last_unstale":"2024-04-15T15:23:56.418641+0000","last_undegraded":"2024-04-15T15:23:56.418641+0000","last_fullsized":"2024-04-15T15:23:56.418641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:52.386885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262108+0000","last_change":"2024-04-15T15:23:54.240249+0000","last_active":"2024-04-15T15:23:56.262108+0000","last_peered":"2024-04-15T15:23:56.262108+0000","last_clean":"2024-04-15T15:23:56.262108+0000","last_became_active":"2024-04-15T15:23:54.239151+0000","last_became_peered":"2024-04-15T15:23:54.239151+0000","last_unstale":"2024-04-15T15:23:56.262108+0000","last_undegraded":"2024-04-15T15:23:56.262108+0000","last_fullsized":"2024-04-15T15:23:56.262108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:53.265693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417969+0000","last_change":"2024-04-15T15:23:55.266293+0000","last_active":"2024-04-15T15:23:56.417969+0000","last_peered":"2024-04-15T15:23:56.417969+0000","last_clean":"2024-04-15T15:23:56.417969+0000","last_became_active":"2024-04-15T15:23:55.265962+0000","last_became_peered":"2024-04-15T15:23:55.265962+0000","last_unstale":"2024-04-15T15:23:56.417969+0000","last_undegraded":"2024-04-15T15:23:56.417969+0000","last_fullsized":"2024-04-15T15:23:56.417969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:40.334427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417723+0000","last_change":"2024-04-15T15:23:54.233459+0000","last_active":"2024-04-15T15:23:56.417723+0000","last_peered":"2024-04-15T15:23:56.417723+0000","last_clean":"2024-04-15T15:23:56.417723+0000","last_became_active":"2024-04-15T15:23:54.233056+0000","last_became_peered":"2024-04-15T15:23:54.233056+0000","last_unstale":"2024-04-15T15:23:56.417723+0000","last_undegraded":"2024-04-15T15:23:56.417723+0000","last_fullsized":"2024-04-15T15:23:56.417723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:38:14.692145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417725+0000","last_change":"2024-04-15T15:23:55.261166+0000","last_active":"2024-04-15T15:23:56.417725+0000","last_peered":"2024-04-15T15:23:56.417725+0000","last_clean":"2024-04-15T15:23:56.417725+0000","last_became_active":"2024-04-15T15:23:55.260827+0000","last_became_peered":"2024-04-15T15:23:55.260827+0000","last_unstale":"2024-04-15T15:23:56.417725+0000","last_undegraded":"2024-04-15T15:23:56.417725+0000","last_fullsized":"2024-04-15T15:23:56.417725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262944+0000","last_change":"2024-04-15T15:23:54.233681+0000","last_active":"2024-04-15T15:23:56.262944+0000","last_peered":"2024-04-15T15:23:56.262944+0000","last_clean":"2024-04-15T15:23:56.262944+0000","last_became_active":"2024-04-15T15:23:54.233376+0000","last_became_peered":"2024-04-15T15:23:54.233376+0000","last_unstale":"2024-04-15T15:23:56.262944+0000","last_undegraded":"2024-04-15T15:23:56.262944+0000","last_fullsized":"2024-04-15T15:23:56.262944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419714+0000","last_change":"2024-04-15T15:23:55.275827+0000","last_active":"2024-04-15T15:23:56.419714+0000","last_peered":"2024-04-15T15:23:56.419714+0000","last_clean":"2024-04-15T15:23:56.419714+0000","last_became_active":"2024-04-15T15:23:55.275511+0000","last_became_peered":"2024-04-15T15:23:55.275511+0000","last_unstale":"2024-04-15T15:23:56.419714+0000","last_undegraded":"2024-04-15T15:23:56.419714+0000","last_fullsized":"2024-04-15T15:23:56.419714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:17:08.438022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261296+0000","last_change":"2024-04-15T15:23:54.237666+0000","last_active":"2024-04-15T15:23:56.261296+0000","last_peered":"2024-04-15T15:23:56.261296+0000","last_clean":"2024-04-15T15:23:56.261296+0000","last_became_active":"2024-04-15T15:23:54.237243+0000","last_became_peered":"2024-04-15T15:23:54.237243+0000","last_unstale":"2024-04-15T15:23:56.261296+0000","last_undegraded":"2024-04-15T15:23:56.261296+0000","last_fullsized":"2024-04-15T15:23:56.261296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262905+0000","last_change":"2024-04-15T15:23:55.264795+0000","last_active":"2024-04-15T15:23:56.262905+0000","last_peered":"2024-04-15T15:23:56.262905+0000","last_clean":"2024-04-15T15:23:56.262905+0000","last_became_active":"2024-04-15T15:23:55.264509+0000","last_became_peered":"2024-04-15T15:23:55.264509+0000","last_unstale":"2024-04-15T15:23:56.262905+0000","last_undegraded":"2024-04-15T15:23:56.262905+0000","last_fullsized":"2024-04-15T15:23:56.262905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416505+0000","last_change":"2024-04-15T15:23:54.233260+0000","last_active":"2024-04-15T15:23:56.416505+0000","last_peered":"2024-04-15T15:23:56.416505+0000","last_clean":"2024-04-15T15:23:56.416505+0000","last_became_active":"2024-04-15T15:23:54.231670+0000","last_became_peered":"2024-04-15T15:23:54.231670+0000","last_unstale":"2024-04-15T15:23:56.416505+0000","last_undegraded":"2024-04-15T15:23:56.416505+0000","last_fullsized":"2024-04-15T15:23:56.416505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:33:24.512549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418111+0000","last_change":"2024-04-15T15:23:55.278880+0000","last_active":"2024-04-15T15:23:56.418111+0000","last_peered":"2024-04-15T15:23:56.418111+0000","last_clean":"2024-04-15T15:23:56.418111+0000","last_became_active":"2024-04-15T15:23:55.278617+0000","last_became_peered":"2024-04-15T15:23:55.278617+0000","last_unstale":"2024-04-15T15:23:56.418111+0000","last_undegraded":"2024-04-15T15:23:56.418111+0000","last_fullsized":"2024-04-15T15:23:56.418111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:16:50.418929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418140+0000","last_change":"2024-04-15T15:23:54.239459+0000","last_active":"2024-04-15T15:23:56.418140+0000","last_peered":"2024-04-15T15:23:56.418140+0000","last_clean":"2024-04-15T15:23:56.418140+0000","last_became_active":"2024-04-15T15:23:54.239033+0000","last_became_peered":"2024-04-15T15:23:54.239033+0000","last_unstale":"2024-04-15T15:23:56.418140+0000","last_undegraded":"2024-04-15T15:23:56.418140+0000","last_fullsized":"2024-04-15T15:23:56.418140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:16.311793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419071+0000","last_change":"2024-04-15T15:23:55.277191+0000","last_active":"2024-04-15T15:23:56.419071+0000","last_peered":"2024-04-15T15:23:56.419071+0000","last_clean":"2024-04-15T15:23:56.419071+0000","last_became_active":"2024-04-15T15:23:55.276803+0000","last_became_peered":"2024-04-15T15:23:55.276803+0000","last_unstale":"2024-04-15T15:23:56.419071+0000","last_undegraded":"2024-04-15T15:23:56.419071+0000","last_fullsized":"2024-04-15T15:23:56.419071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:53:48.793142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417566+0000","last_change":"2024-04-15T15:23:54.230441+0000","last_active":"2024-04-15T15:23:56.417566+0000","last_peered":"2024-04-15T15:23:56.417566+0000","last_clean":"2024-04-15T15:23:56.417566+0000","last_became_active":"2024-04-15T15:23:54.230051+0000","last_became_peered":"2024-04-15T15:23:54.230051+0000","last_unstale":"2024-04-15T15:23:56.417566+0000","last_undegraded":"2024-04-15T15:23:56.417566+0000","last_fullsized":"2024-04-15T15:23:56.417566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:29.938678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418932+0000","last_change":"2024-04-15T15:23:55.281762+0000","last_active":"2024-04-15T15:23:56.418932+0000","last_peered":"2024-04-15T15:23:56.418932+0000","last_clean":"2024-04-15T15:23:56.418932+0000","last_became_active":"2024-04-15T15:23:55.281494+0000","last_became_peered":"2024-04-15T15:23:55.281494+0000","last_unstale":"2024-04-15T15:23:56.418932+0000","last_undegraded":"2024-04-15T15:23:56.418932+0000","last_fullsized":"2024-04-15T15:23:56.418932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:42:09.480734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255780+0000","last_change":"2024-04-15T15:23:54.234786+0000","last_active":"2024-04-15T15:23:56.255780+0000","last_peered":"2024-04-15T15:23:56.255780+0000","last_clean":"2024-04-15T15:23:56.255780+0000","last_became_active":"2024-04-15T15:23:54.234484+0000","last_became_peered":"2024-04-15T15:23:54.234484+0000","last_unstale":"2024-04-15T15:23:56.255780+0000","last_undegraded":"2024-04-15T15:23:56.255780+0000","last_fullsized":"2024-04-15T15:23:56.255780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:05.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262958+0000","last_change":"2024-04-15T15:23:55.267889+0000","last_active":"2024-04-15T15:23:56.262958+0000","last_peered":"2024-04-15T15:23:56.262958+0000","last_clean":"2024-04-15T15:23:56.262958+0000","last_became_active":"2024-04-15T15:23:55.267209+0000","last_became_peered":"2024-04-15T15:23:55.267209+0000","last_unstale":"2024-04-15T15:23:56.262958+0000","last_undegraded":"2024-04-15T15:23:56.262958+0000","last_fullsized":"2024-04-15T15:23:56.262958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:40.334427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416588+0000","last_change":"2024-04-15T15:23:54.237963+0000","last_active":"2024-04-15T15:23:56.416588+0000","last_peered":"2024-04-15T15:23:56.416588+0000","last_clean":"2024-04-15T15:23:56.416588+0000","last_became_active":"2024-04-15T15:23:54.237726+0000","last_became_peered":"2024-04-15T15:23:54.237726+0000","last_unstale":"2024-04-15T15:23:56.416588+0000","last_undegraded":"2024-04-15T15:23:56.416588+0000","last_fullsized":"2024-04-15T15:23:56.416588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:45:06.635874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418333+0000","last_change":"2024-04-15T15:23:55.272911+0000","last_active":"2024-04-15T15:23:56.418333+0000","last_peered":"2024-04-15T15:23:56.418333+0000","last_clean":"2024-04-15T15:23:56.418333+0000","last_became_active":"2024-04-15T15:23:55.272642+0000","last_became_peered":"2024-04-15T15:23:55.272642+0000","last_unstale":"2024-04-15T15:23:56.418333+0000","last_undegraded":"2024-04-15T15:23:56.418333+0000","last_fullsized":"2024-04-15T15:23:56.418333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:27.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417306+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:23:56.417306+0000","last_peered":"2024-04-15T15:23:56.417306+0000","last_clean":"2024-04-15T15:23:56.417306+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:23:56.417306+0000","last_undegraded":"2024-04-15T15:23:56.417306+0000","last_fullsized":"2024-04-15T15:23:56.417306+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261029+0000","last_change":"2024-04-15T15:23:54.232974+0000","last_active":"2024-04-15T15:23:56.261029+0000","last_peered":"2024-04-15T15:23:56.261029+0000","last_clean":"2024-04-15T15:23:56.261029+0000","last_became_active":"2024-04-15T15:23:54.230808+0000","last_became_peered":"2024-04-15T15:23:54.230808+0000","last_unstale":"2024-04-15T15:23:56.261029+0000","last_undegraded":"2024-04-15T15:23:56.261029+0000","last_fullsized":"2024-04-15T15:23:56.261029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:46:30.058785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418655+0000","last_change":"2024-04-15T15:23:55.269058+0000","last_active":"2024-04-15T15:23:56.418655+0000","last_peered":"2024-04-15T15:23:56.418655+0000","last_clean":"2024-04-15T15:23:56.418655+0000","last_became_active":"2024-04-15T15:23:55.268267+0000","last_became_peered":"2024-04-15T15:23:55.268267+0000","last_unstale":"2024-04-15T15:23:56.418655+0000","last_undegraded":"2024-04-15T15:23:56.418655+0000","last_fullsized":"2024-04-15T15:23:56.418655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:30:18.931762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417047+0000","last_change":"2024-04-15T15:23:54.251259+0000","last_active":"2024-04-15T15:23:56.417047+0000","last_peered":"2024-04-15T15:23:56.417047+0000","last_clean":"2024-04-15T15:23:56.417047+0000","last_became_active":"2024-04-15T15:23:54.250985+0000","last_became_peered":"2024-04-15T15:23:54.250985+0000","last_unstale":"2024-04-15T15:23:56.417047+0000","last_undegraded":"2024-04-15T15:23:56.417047+0000","last_fullsized":"2024-04-15T15:23:56.417047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:44:11.419953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418428+0000","last_change":"2024-04-15T15:23:55.267279+0000","last_active":"2024-04-15T15:23:56.418428+0000","last_peered":"2024-04-15T15:23:56.418428+0000","last_clean":"2024-04-15T15:23:56.418428+0000","last_became_active":"2024-04-15T15:23:55.266971+0000","last_became_peered":"2024-04-15T15:23:55.266971+0000","last_unstale":"2024-04-15T15:23:56.418428+0000","last_undegraded":"2024-04-15T15:23:56.418428+0000","last_fullsized":"2024-04-15T15:23:56.418428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417318+0000","last_change":"2024-04-15T15:23:54.234761+0000","last_active":"2024-04-15T15:23:56.417318+0000","last_peered":"2024-04-15T15:23:56.417318+0000","last_clean":"2024-04-15T15:23:56.417318+0000","last_became_active":"2024-04-15T15:23:54.234467+0000","last_became_peered":"2024-04-15T15:23:54.234467+0000","last_unstale":"2024-04-15T15:23:56.417318+0000","last_undegraded":"2024-04-15T15:23:56.417318+0000","last_fullsized":"2024-04-15T15:23:56.417318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:47:42.194686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255347+0000","last_change":"2024-04-15T15:23:55.244630+0000","last_active":"2024-04-15T15:23:56.255347+0000","last_peered":"2024-04-15T15:23:56.255347+0000","last_clean":"2024-04-15T15:23:56.255347+0000","last_became_active":"2024-04-15T15:23:55.244208+0000","last_became_peered":"2024-04-15T15:23:55.244208+0000","last_unstale":"2024-04-15T15:23:56.255347+0000","last_undegraded":"2024-04-15T15:23:56.255347+0000","last_fullsized":"2024-04-15T15:23:56.255347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:08:54.112562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263182+0000","last_change":"2024-04-15T15:23:54.225774+0000","last_active":"2024-04-15T15:23:56.263182+0000","last_peered":"2024-04-15T15:23:56.263182+0000","last_clean":"2024-04-15T15:23:56.263182+0000","last_became_active":"2024-04-15T15:23:54.223951+0000","last_became_peered":"2024-04-15T15:23:54.223951+0000","last_unstale":"2024-04-15T15:23:56.263182+0000","last_undegraded":"2024-04-15T15:23:56.263182+0000","last_fullsized":"2024-04-15T15:23:56.263182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417834+0000","last_change":"2024-04-15T15:23:55.261370+0000","last_active":"2024-04-15T15:23:56.417834+0000","last_peered":"2024-04-15T15:23:56.417834+0000","last_clean":"2024-04-15T15:23:56.417834+0000","last_became_active":"2024-04-15T15:23:55.260930+0000","last_became_peered":"2024-04-15T15:23:55.260930+0000","last_unstale":"2024-04-15T15:23:56.417834+0000","last_undegraded":"2024-04-15T15:23:56.417834+0000","last_fullsized":"2024-04-15T15:23:56.417834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:32:54.270601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255534+0000","last_change":"2024-04-15T15:23:54.223225+0000","last_active":"2024-04-15T15:23:56.255534+0000","last_peered":"2024-04-15T15:23:56.255534+0000","last_clean":"2024-04-15T15:23:56.255534+0000","last_became_active":"2024-04-15T15:23:54.222903+0000","last_became_peered":"2024-04-15T15:23:54.222903+0000","last_unstale":"2024-04-15T15:23:56.255534+0000","last_undegraded":"2024-04-15T15:23:56.255534+0000","last_fullsized":"2024-04-15T15:23:56.255534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:21:51.381976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.264575+0000","last_change":"2024-04-15T15:23:55.270383+0000","last_active":"2024-04-15T15:23:56.264575+0000","last_peered":"2024-04-15T15:23:56.264575+0000","last_clean":"2024-04-15T15:23:56.264575+0000","last_became_active":"2024-04-15T15:23:55.267157+0000","last_became_peered":"2024-04-15T15:23:55.267157+0000","last_unstale":"2024-04-15T15:23:56.264575+0000","last_undegraded":"2024-04-15T15:23:56.264575+0000","last_fullsized":"2024-04-15T15:23:56.264575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:41:27.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419141+0000","last_change":"2024-04-15T15:23:54.251497+0000","last_active":"2024-04-15T15:23:56.419141+0000","last_peered":"2024-04-15T15:23:56.419141+0000","last_clean":"2024-04-15T15:23:56.419141+0000","last_became_active":"2024-04-15T15:23:54.250660+0000","last_became_peered":"2024-04-15T15:23:54.250660+0000","last_unstale":"2024-04-15T15:23:56.419141+0000","last_undegraded":"2024-04-15T15:23:56.419141+0000","last_fullsized":"2024-04-15T15:23:56.419141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:14.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255397+0000","last_change":"2024-04-15T15:23:55.249702+0000","last_active":"2024-04-15T15:23:56.255397+0000","last_peered":"2024-04-15T15:23:56.255397+0000","last_clean":"2024-04-15T15:23:56.255397+0000","last_became_active":"2024-04-15T15:23:55.249084+0000","last_became_peered":"2024-04-15T15:23:55.249084+0000","last_unstale":"2024-04-15T15:23:56.255397+0000","last_undegraded":"2024-04-15T15:23:56.255397+0000","last_fullsized":"2024-04-15T15:23:56.255397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:58:34.912829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255845+0000","last_change":"2024-04-15T15:23:54.241520+0000","last_active":"2024-04-15T15:23:56.255845+0000","last_peered":"2024-04-15T15:23:56.255845+0000","last_clean":"2024-04-15T15:23:56.255845+0000","last_became_active":"2024-04-15T15:23:54.241103+0000","last_became_peered":"2024-04-15T15:23:54.241103+0000","last_unstale":"2024-04-15T15:23:56.255845+0000","last_undegraded":"2024-04-15T15:23:56.255845+0000","last_fullsized":"2024-04-15T15:23:56.255845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:44:11.419953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415819+0000","last_change":"2024-04-15T15:23:55.265567+0000","last_active":"2024-04-15T15:23:56.415819+0000","last_peered":"2024-04-15T15:23:56.415819+0000","last_clean":"2024-04-15T15:23:56.415819+0000","last_became_active":"2024-04-15T15:23:55.262905+0000","last_became_peered":"2024-04-15T15:23:55.262905+0000","last_unstale":"2024-04-15T15:23:56.415819+0000","last_undegraded":"2024-04-15T15:23:56.415819+0000","last_fullsized":"2024-04-15T15:23:56.415819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:06:05.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.263461+0000","last_change":"2024-04-15T15:23:54.252083+0000","last_active":"2024-04-15T15:23:56.263461+0000","last_peered":"2024-04-15T15:23:56.263461+0000","last_clean":"2024-04-15T15:23:56.263461+0000","last_became_active":"2024-04-15T15:23:54.251796+0000","last_became_peered":"2024-04-15T15:23:54.251796+0000","last_unstale":"2024-04-15T15:23:56.263461+0000","last_undegraded":"2024-04-15T15:23:56.263461+0000","last_fullsized":"2024-04-15T15:23:56.263461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:46:30.058785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416720+0000","last_change":"2024-04-15T15:23:55.257995+0000","last_active":"2024-04-15T15:23:56.416720+0000","last_peered":"2024-04-15T15:23:56.416720+0000","last_clean":"2024-04-15T15:23:56.416720+0000","last_became_active":"2024-04-15T15:23:55.257718+0000","last_became_peered":"2024-04-15T15:23:55.257718+0000","last_unstale":"2024-04-15T15:23:56.416720+0000","last_undegraded":"2024-04-15T15:23:56.416720+0000","last_fullsized":"2024-04-15T15:23:56.416720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:08:29.731069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416748+0000","last_change":"2024-04-15T15:23:54.250977+0000","last_active":"2024-04-15T15:23:56.416748+0000","last_peered":"2024-04-15T15:23:56.416748+0000","last_clean":"2024-04-15T15:23:56.416748+0000","last_became_active":"2024-04-15T15:23:54.250302+0000","last_became_peered":"2024-04-15T15:23:54.250302+0000","last_unstale":"2024-04-15T15:23:56.416748+0000","last_undegraded":"2024-04-15T15:23:56.416748+0000","last_fullsized":"2024-04-15T15:23:56.416748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:05.894114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418747+0000","last_change":"2024-04-15T15:23:55.260220+0000","last_active":"2024-04-15T15:23:56.418747+0000","last_peered":"2024-04-15T15:23:56.418747+0000","last_clean":"2024-04-15T15:23:56.418747+0000","last_became_active":"2024-04-15T15:23:55.259361+0000","last_became_peered":"2024-04-15T15:23:55.259361+0000","last_unstale":"2024-04-15T15:23:56.418747+0000","last_undegraded":"2024-04-15T15:23:56.418747+0000","last_fullsized":"2024-04-15T15:23:56.418747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:45:06.899022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255405+0000","last_change":"2024-04-15T15:23:54.236256+0000","last_active":"2024-04-15T15:23:56.255405+0000","last_peered":"2024-04-15T15:23:56.255405+0000","last_clean":"2024-04-15T15:23:56.255405+0000","last_became_active":"2024-04-15T15:23:54.235919+0000","last_became_peered":"2024-04-15T15:23:54.235919+0000","last_unstale":"2024-04-15T15:23:56.255405+0000","last_undegraded":"2024-04-15T15:23:56.255405+0000","last_fullsized":"2024-04-15T15:23:56.255405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:10:31.149583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415070+0000","last_change":"2024-04-15T15:23:55.258682+0000","last_active":"2024-04-15T15:23:56.415070+0000","last_peered":"2024-04-15T15:23:56.415070+0000","last_clean":"2024-04-15T15:23:56.415070+0000","last_became_active":"2024-04-15T15:23:55.258183+0000","last_became_peered":"2024-04-15T15:23:55.258183+0000","last_unstale":"2024-04-15T15:23:56.415070+0000","last_undegraded":"2024-04-15T15:23:56.415070+0000","last_fullsized":"2024-04-15T15:23:56.415070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.255301+0000","last_change":"2024-04-15T15:23:54.233682+0000","last_active":"2024-04-15T15:23:56.255301+0000","last_peered":"2024-04-15T15:23:56.255301+0000","last_clean":"2024-04-15T15:23:56.255301+0000","last_became_active":"2024-04-15T15:23:54.233373+0000","last_became_peered":"2024-04-15T15:23:54.233373+0000","last_unstale":"2024-04-15T15:23:56.255301+0000","last_undegraded":"2024-04-15T15:23:56.255301+0000","last_fullsized":"2024-04-15T15:23:56.255301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:00:24.931262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416048+0000","last_change":"2024-04-15T15:23:55.272358+0000","last_active":"2024-04-15T15:23:56.416048+0000","last_peered":"2024-04-15T15:23:56.416048+0000","last_clean":"2024-04-15T15:23:56.416048+0000","last_became_active":"2024-04-15T15:23:55.271660+0000","last_became_peered":"2024-04-15T15:23:55.271660+0000","last_unstale":"2024-04-15T15:23:56.416048+0000","last_undegraded":"2024-04-15T15:23:56.416048+0000","last_fullsized":"2024-04-15T15:23:56.416048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:18:18.111972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.419745+0000","last_change":"2024-04-15T15:23:54.251693+0000","last_active":"2024-04-15T15:23:56.419745+0000","last_peered":"2024-04-15T15:23:56.419745+0000","last_clean":"2024-04-15T15:23:56.419745+0000","last_became_active":"2024-04-15T15:23:54.250255+0000","last_became_peered":"2024-04-15T15:23:54.250255+0000","last_unstale":"2024-04-15T15:23:56.419745+0000","last_undegraded":"2024-04-15T15:23:56.419745+0000","last_fullsized":"2024-04-15T15:23:56.419745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:08:28.726161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.261853+0000","last_change":"2024-04-15T15:23:55.278856+0000","last_active":"2024-04-15T15:23:56.261853+0000","last_peered":"2024-04-15T15:23:56.261853+0000","last_clean":"2024-04-15T15:23:56.261853+0000","last_became_active":"2024-04-15T15:23:55.278605+0000","last_became_peered":"2024-04-15T15:23:55.278605+0000","last_unstale":"2024-04-15T15:23:56.261853+0000","last_undegraded":"2024-04-15T15:23:56.261853+0000","last_fullsized":"2024-04-15T15:23:56.261853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:48:51.030877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.264390+0000","last_change":"2024-04-15T15:23:55.260285+0000","last_active":"2024-04-15T15:23:56.264390+0000","last_peered":"2024-04-15T15:23:56.264390+0000","last_clean":"2024-04-15T15:23:56.264390+0000","last_became_active":"2024-04-15T15:23:55.260016+0000","last_became_peered":"2024-04-15T15:23:55.260016+0000","last_unstale":"2024-04-15T15:23:56.264390+0000","last_undegraded":"2024-04-15T15:23:56.264390+0000","last_fullsized":"2024-04-15T15:23:56.264390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:16:58.465465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418779+0000","last_change":"2024-04-15T15:23:54.250621+0000","last_active":"2024-04-15T15:23:56.418779+0000","last_peered":"2024-04-15T15:23:56.418779+0000","last_clean":"2024-04-15T15:23:56.418779+0000","last_became_active":"2024-04-15T15:23:54.250204+0000","last_became_peered":"2024-04-15T15:23:54.250204+0000","last_unstale":"2024-04-15T15:23:56.418779+0000","last_undegraded":"2024-04-15T15:23:56.418779+0000","last_fullsized":"2024-04-15T15:23:56.418779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:54:51.507224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.262544+0000","last_change":"2024-04-15T15:23:55.263054+0000","last_active":"2024-04-15T15:23:56.262544+0000","last_peered":"2024-04-15T15:23:56.262544+0000","last_clean":"2024-04-15T15:23:56.262544+0000","last_became_active":"2024-04-15T15:23:55.262742+0000","last_became_peered":"2024-04-15T15:23:55.262742+0000","last_unstale":"2024-04-15T15:23:56.262544+0000","last_undegraded":"2024-04-15T15:23:56.262544+0000","last_fullsized":"2024-04-15T15:23:56.262544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:30:18.931762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.418457+0000","last_change":"2024-04-15T15:23:54.250572+0000","last_active":"2024-04-15T15:23:56.418457+0000","last_peered":"2024-04-15T15:23:56.418457+0000","last_clean":"2024-04-15T15:23:56.418457+0000","last_became_active":"2024-04-15T15:23:54.250229+0000","last_became_peered":"2024-04-15T15:23:54.250229+0000","last_unstale":"2024-04-15T15:23:56.418457+0000","last_undegraded":"2024-04-15T15:23:56.418457+0000","last_fullsized":"2024-04-15T15:23:56.418457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:15.052794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.416985+0000","last_change":"2024-04-15T15:23:55.269263+0000","last_active":"2024-04-15T15:23:56.416985+0000","last_peered":"2024-04-15T15:23:56.416985+0000","last_clean":"2024-04-15T15:23:56.416985+0000","last_became_active":"2024-04-15T15:23:55.268706+0000","last_became_peered":"2024-04-15T15:23:55.268706+0000","last_unstale":"2024-04-15T15:23:56.416985+0000","last_undegraded":"2024-04-15T15:23:56.416985+0000","last_fullsized":"2024-04-15T15:23:56.416985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:05:16.057703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415538+0000","last_change":"2024-04-15T15:23:54.229794+0000","last_active":"2024-04-15T15:23:56.415538+0000","last_peered":"2024-04-15T15:23:56.415538+0000","last_clean":"2024-04-15T15:23:56.415538+0000","last_became_active":"2024-04-15T15:23:54.229509+0000","last_became_peered":"2024-04-15T15:23:54.229509+0000","last_unstale":"2024-04-15T15:23:56.415538+0000","last_undegraded":"2024-04-15T15:23:56.415538+0000","last_fullsized":"2024-04-15T15:23:56.415538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:49:04.203638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.415289+0000","last_change":"2024-04-15T15:23:55.274587+0000","last_active":"2024-04-15T15:23:56.415289+0000","last_peered":"2024-04-15T15:23:56.415289+0000","last_clean":"2024-04-15T15:23:56.415289+0000","last_became_active":"2024-04-15T15:23:55.274127+0000","last_became_peered":"2024-04-15T15:23:55.274127+0000","last_unstale":"2024-04-15T15:23:56.415289+0000","last_undegraded":"2024-04-15T15:23:56.415289+0000","last_fullsized":"2024-04-15T15:23:56.415289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:54.193276+0000","last_clean_scrub_stamp":"2024-04-15T15:23:54.193276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:57:40.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-15T15:23:56.417220+0000","last_change":"2024-04-15T15:23:54.236884+0000","last_active":"2024-04-15T15:23:56.417220+0000","last_peered":"2024-04-15T15:23:56.417220+0000","last_clean":"2024-04-15T15:23:56.417220+0000","last_became_active":"2024-04-15T15:23:54.236583+0000","last_became_peered":"2024-04-15T15:23:54.236583+0000","last_unstale":"2024-04-15T15:23:56.417220+0000","last_undegraded":"2024-04-15T15:23:56.417220+0000","last_fullsized":"2024-04-15T15:23:56.417220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:53.188368+0000","last_clean_scrub_stamp":"2024-04-15T15:23:53.188368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:50:33.748557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":667234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":76410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":667290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:24:05.793 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-15T15:24:05.793 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-15T15:24:05.793 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-15T15:24:05.793 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-15T15:24:06.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.037+0000 7f5819600640 1 Processor -- start 2024-04-15T15:24:06.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.038+0000 7f5819600640 1 -- start start 2024-04-15T15:24:06.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.039+0000 7f5819600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f581415b860 0x7f581406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.039+0000 7f5819600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140cdfb0 con 0x7f581406f320 2024-04-15T15:24:06.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5819600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f581415f2b0 con 0x7f581415b480 2024-04-15T15:24:06.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5819600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f581415f060 con 0x7f581415b860 2024-04-15T15:24:06.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5813400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f581415b860 0x7f581406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5818c00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f581406f320 0x7f58140b3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:33564/0 (socket says 172.21.15.150:33564) 2024-04-15T15:24:06.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5818c00640 1 -- 172.21.15.150:0/3800908507 learned_addr learned my addr 172.21.15.150:0/3800908507 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:06.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5813400640 1 -- 172.21.15.150:0/3800908507 >> v1:172.21.15.187:6789/0 conn(0x7f581415b480 legacy=0x7f58140cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5813400640 1 -- 172.21.15.150:0/3800908507 >> v1:172.21.15.150:6789/0 conn(0x7f581406f320 legacy=0x7f58140b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5813400640 1 -- 172.21.15.150:0/3800908507 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581415f590 con 0x7f581415b860 2024-04-15T15:24:06.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.040+0000 7f5812a00640 1 -- 172.21.15.150:0/3800908507 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1843482861 0 0) 0x7f58140cdfb0 con 0x7f581406f320 2024-04-15T15:24:06.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.041+0000 7f5813400640 1 --2- 172.21.15.150:0/3800908507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581415b860 0x7f581406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009d90 tx=0x7f57fc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a9706261806110 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.042+0000 7f5812a00640 1 -- 172.21.15.150:0/3800908507 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc039070 con 0x7f581415b860 2024-04-15T15:24:06.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.042+0000 7f5812a00640 1 -- 172.21.15.150:0/3800908507 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc02cd00 con 0x7f581415b860 2024-04-15T15:24:06.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.042+0000 7f5812a00640 1 -- 172.21.15.150:0/3800908507 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc034660 con 0x7f581415b860 2024-04-15T15:24:06.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.042+0000 7f5819600640 1 -- 172.21.15.150:0/3800908507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581415b860 msgr2=0x7f581406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.042+0000 7f5819600640 1 --2- 172.21.15.150:0/3800908507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581415b860 0x7f581406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009d90 tx=0x7f57fc02bf50 comp rx=0 tx=0).stop 2024-04-15T15:24:06.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.043+0000 7f5819600640 1 -- 172.21.15.150:0/3800908507 shutdown_connections 2024-04-15T15:24:06.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.043+0000 7f5819600640 1 --2- 172.21.15.150:0/3800908507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581415b860 0x7f581406ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.043+0000 7f5819600640 1 -- 172.21.15.150:0/3800908507 >> 172.21.15.150:0/3800908507 conn(0x7f581406b930 msgr2=0x7f581406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:06.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.043+0000 7f5819600640 1 -- 172.21.15.150:0/3800908507 shutdown_connections 2024-04-15T15:24:06.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.043+0000 7f5819600640 1 -- 172.21.15.150:0/3800908507 wait complete. 2024-04-15T15:24:06.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.045+0000 7f5819600640 1 Processor -- start 2024-04-15T15:24:06.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.045+0000 7f5819600640 1 -- start start 2024-04-15T15:24:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5819600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581406f320 0x7f581414dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5819600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f581415b480 0x7f5814140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5819600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f581415b860 0x7f5814151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5819600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f581415fb50 con 0x7f581406f320 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5819600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f581415fec0 con 0x7f581415b480 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5819600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f58140cd550 con 0x7f581415b860 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581406f320 0x7f581414dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581406f320 0x7f581414dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44818/0 (socket says 172.21.15.150:44818) 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 -- 172.21.15.150:0/3337977851 learned_addr learned my addr 172.21.15.150:0/3337977851 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 -- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f581415b860 msgr2=0x7f5814151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f581415b860 0x7f5814151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 -- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f581415b480 msgr2=0x7f5814140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f581415b480 0x7f5814140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 -- 172.21.15.150:0/3337977851 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5814151c80 con 0x7f581406f320 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.046+0000 7f5813e00640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581406f320 0x7f581414dd10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f580400c480 tx=0x7f580400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.047+0000 7f580be00640 1 -- 172.21.15.150:0/3337977851 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040052d0 con 0x7f581406f320 2024-04-15T15:24:06.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.047+0000 7f580be00640 1 -- 172.21.15.150:0/3337977851 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804005470 con 0x7f581406f320 2024-04-15T15:24:06.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.047+0000 7f580be00640 1 -- 172.21.15.150:0/3337977851 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804012870 con 0x7f581406f320 2024-04-15T15:24:06.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.047+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581414e360 con 0x7f581406f320 2024-04-15T15:24:06.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.047+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581414e920 con 0x7f581406f320 2024-04-15T15:24:06.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.048+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58140755c0 con 0x7f581406f320 2024-04-15T15:24:06.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.049+0000 7f580be00640 1 -- 172.21.15.150:0/3337977851 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5804012a10 con 0x7f581406f320 2024-04-15T15:24:06.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.050+0000 7f580be00640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57f0070d00 0x7f57f00731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.050+0000 7f580be00640 1 -- 172.21.15.150:0/3337977851 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f580408b800 con 0x7f581406f320 2024-04-15T15:24:06.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.051+0000 7f5813400640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57f0070d00 0x7f57f00731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.052+0000 7f5813400640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57f0070d00 0x7f57f00731c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f57fc036000 tx=0x7f57fc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.055+0000 7f580be00640 1 -- 172.21.15.150:0/3337977851 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5804059c20 con 0x7f581406f320 2024-04-15T15:24:06.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.296+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f58140b3930 con 0x7f581406f320 2024-04-15T15:24:06.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.296+0000 7f580be00640 1 -- 172.21.15.150:0/3337977851 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5804059520 con 0x7f581406f320 2024-04-15T15:24:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:06.298 INFO:teuthology.orchestra.run.smithi150.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57f0070d00 msgr2=0x7f57f00731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57f0070d00 0x7f57f00731c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f57fc036000 tx=0x7f57fc02e000 comp rx=0 tx=0).stop 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581406f320 msgr2=0x7f581414dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581406f320 0x7f581414dd10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f580400c480 tx=0x7f580400c940 comp rx=0 tx=0).stop 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 shutdown_connections 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57f0070d00 0x7f57f00731c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f581415b860 0x7f5814151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f581415b480 0x7f5814140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 --2- 172.21.15.150:0/3337977851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f581406f320 0x7f581414dd10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 >> 172.21.15.150:0/3337977851 conn(0x7f581406b930 msgr2=0x7f58140b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:06.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 shutdown_connections 2024-04-15T15:24:06.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.299+0000 7f5819600640 1 -- 172.21.15.150:0/3337977851 wait complete. 2024-04-15T15:24:06.322 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-15T15:24:06.323 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:06.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.513+0000 7fea0e000640 1 Processor -- start 2024-04-15T15:24:06.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.513+0000 7fea0e000640 1 -- start start 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.515+0000 7fea0e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea080cee00 0x7fea0806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.515+0000 7fea0e000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fea08071f50 0x7fea080723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.515+0000 7fea0e000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea0815dbb0 con 0x7fea0815cbe0 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.515+0000 7fea0e000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fea0806f4f0 con 0x7fea08071f50 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.515+0000 7fea0e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea080728f0 con 0x7fea080cee00 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.515+0000 7fea0cc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea080cee00 0x7fea0806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.515+0000 7fea0cc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea080cee00 0x7fea0806e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:41730/0 (socket says 172.21.15.150:41730) 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.515+0000 7fea07e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fea08071f50 0x7fea080723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.516+0000 7fea0cc00640 1 -- 172.21.15.150:0/3466956985 learned_addr learned my addr 172.21.15.150:0/3466956985 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:06.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.516+0000 7fea07e00640 1 -- 172.21.15.150:0/3466956985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea080cee00 msgr2=0x7fea0806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:06.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.516+0000 7fea07e00640 1 --2- 172.21.15.150:0/3466956985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea080cee00 0x7fea0806e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.516+0000 7fea07e00640 1 -- 172.21.15.150:0/3466956985 >> v1:172.21.15.187:6789/0 conn(0x7fea0815cbe0 legacy=0x7fea0815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.516+0000 7fea07e00640 1 -- 172.21.15.150:0/3466956985 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0815d8d0 con 0x7fea08071f50 2024-04-15T15:24:06.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.516+0000 7fea0cc00640 1 --2- 172.21.15.150:0/3466956985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea080cee00 0x7fea0806e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:24:06.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.516+0000 7fea07400640 1 -- 172.21.15.150:0/3466956985 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352939 0 0) 0x7fea0815dbb0 con 0x7fea0815cbe0 2024-04-15T15:24:06.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.518+0000 7fea07e00640 1 --2- 172.21.15.150:0/3466956985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 0x7fea080723b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40097d0 tx=0x7fe9f402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15ee4870c5f31bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.518+0000 7fea07400640 1 -- 172.21.15.150:0/3466956985 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4035020 con 0x7fea08071f50 2024-04-15T15:24:06.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.518+0000 7fea07400640 1 -- 172.21.15.150:0/3466956985 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f4031810 con 0x7fea08071f50 2024-04-15T15:24:06.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.518+0000 7fea07400640 1 -- 172.21.15.150:0/3466956985 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4031be0 con 0x7fea08071f50 2024-04-15T15:24:06.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.519+0000 7fea0e000640 1 -- 172.21.15.150:0/3466956985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 msgr2=0x7fea080723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.519+0000 7fea0e000640 1 --2- 172.21.15.150:0/3466956985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 0x7fea080723b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40097d0 tx=0x7fe9f402b8b0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.519+0000 7fea0e000640 1 -- 172.21.15.150:0/3466956985 shutdown_connections 2024-04-15T15:24:06.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.519+0000 7fea0e000640 1 --2- 172.21.15.150:0/3466956985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 0x7fea080723b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.519+0000 7fea0e000640 1 --2- 172.21.15.150:0/3466956985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea080cee00 0x7fea0806e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.519+0000 7fea0e000640 1 -- 172.21.15.150:0/3466956985 >> 172.21.15.150:0/3466956985 conn(0x7fea0806b930 msgr2=0x7fea0806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:06.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.519+0000 7fea0e000640 1 -- 172.21.15.150:0/3466956985 shutdown_connections 2024-04-15T15:24:06.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.519+0000 7fea0e000640 1 -- 172.21.15.150:0/3466956985 wait complete. 2024-04-15T15:24:06.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.521+0000 7fea0e000640 1 Processor -- start 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.522+0000 7fea0e000640 1 -- start start 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea0e000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 0x7fea08154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea0e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea080cee00 0x7fea08149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea0e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea0815cbe0 0x7fea08159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea0e000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea0806e3a0 con 0x7fea080cee00 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea0e000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea0806e0c0 con 0x7fea08071f50 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea0e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea0815dda0 con 0x7fea0815cbe0 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea07e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea080cee00 0x7fea08149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea07e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea080cee00 0x7fea08149fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44830/0 (socket says 172.21.15.150:44830) 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea07e00640 1 -- 172.21.15.150:0/2381068608 learned_addr learned my addr 172.21.15.150:0/2381068608 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea0cc00640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 0x7fea08154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea0d600640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea0815cbe0 0x7fea08159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea07e00640 1 -- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea0815cbe0 msgr2=0x7fea08159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea07e00640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea0815cbe0 0x7fea08159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea07e00640 1 -- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 msgr2=0x7fea08154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea07e00640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 0x7fea08154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.523+0000 7fea07e00640 1 -- 172.21.15.150:0/2381068608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea08157510 con 0x7fea080cee00 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.524+0000 7fea07e00640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea080cee00 0x7fea08149fc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4000c00 tx=0x7fe9f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.524+0000 7fea04c00640 1 -- 172.21.15.150:0/2381068608 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4035050 con 0x7fea080cee00 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.524+0000 7fea04c00640 1 -- 172.21.15.150:0/2381068608 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f403bcf0 con 0x7fea080cee00 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.524+0000 7fea04c00640 1 -- 172.21.15.150:0/2381068608 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4006e10 con 0x7fea080cee00 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.524+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea081554c0 con 0x7fea080cee00 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.524+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea08155a80 con 0x7fea080cee00 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.525+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea080755c0 con 0x7fea080cee00 2024-04-15T15:24:06.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.526+0000 7fea04c00640 1 -- 172.21.15.150:0/2381068608 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9f4038020 con 0x7fea080cee00 2024-04-15T15:24:06.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.527+0000 7fea04c00640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e806c8f0 0x7fe9e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.528+0000 7fea04c00640 1 -- 172.21.15.150:0/2381068608 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe9f40adb00 con 0x7fea080cee00 2024-04-15T15:24:06.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.528+0000 7fea0cc00640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e806c8f0 0x7fe9e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.529+0000 7fea0cc00640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e806c8f0 0x7fe9e806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009b70 tx=0x7fe9fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.532+0000 7fea04c00640 1 -- 172.21.15.150:0/2381068608 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9f4040070 con 0x7fea080cee00 2024-04-15T15:24:06.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.723+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea08068b90 con 0x7fea080cee00 2024-04-15T15:24:06.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.724+0000 7fea04c00640 1 -- 172.21.15.150:0/2381068608 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7fe9f40ade20 con 0x7fea080cee00 2024-04-15T15:24:06.726 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:06.726 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:06.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e806c8f0 msgr2=0x7fe9e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e806c8f0 0x7fe9e806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009b70 tx=0x7fe9fc008040 comp rx=0 tx=0).stop 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea080cee00 msgr2=0x7fea08149fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea080cee00 0x7fea08149fc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4000c00 tx=0x7fe9f40046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 shutdown_connections 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e806c8f0 0x7fe9e806edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea0815cbe0 0x7fea08159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea080cee00 0x7fea08149fc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 --2- 172.21.15.150:0/2381068608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea08071f50 0x7fea08154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 >> 172.21.15.150:0/2381068608 conn(0x7fea0806b930 msgr2=0x7fea0815bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 shutdown_connections 2024-04-15T15:24:06.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.727+0000 7fea0e000640 1 -- 172.21.15.150:0/2381068608 wait complete. 2024-04-15T15:24:06.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:06.747 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:06.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.951+0000 7f28fcc00640 1 Processor -- start 2024-04-15T15:24:06.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.951+0000 7f28fcc00640 1 -- start start 2024-04-15T15:24:06.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.952+0000 7f28fcc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f815a440 0x7f28f815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.952+0000 7f28fcc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28f8072150 0x7f28f80725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.952+0000 7f28fcc00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f815d520 con 0x7f28f806fc80 2024-04-15T15:24:06.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.953+0000 7f28fcc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f28f8072b00 con 0x7f28f8072150 2024-04-15T15:24:06.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.953+0000 7f28fcc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28f806e280 con 0x7f28f815a440 2024-04-15T15:24:06.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.953+0000 7f28efe00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f28f806fc80 0x7f28f80cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42186/0 (socket says 172.21.15.150:42186) 2024-04-15T15:24:06.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.953+0000 7f28efe00640 1 -- 172.21.15.150:0/563668397 learned_addr learned my addr 172.21.15.150:0/563668397 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.953+0000 7f28f5e00640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f815a440 0x7f28f815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.953+0000 7f28f6800640 1 --2- 172.21.15.150:0/563668397 >> v2:172.21.15.187:3300/0 conn(0x7f28f8072150 0x7f28f80725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.953+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270880337 0 0) 0x7f28f815d520 con 0x7f28f806fc80 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.953+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28dc003520 con 0x7f28f806fc80 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2106800165 0 0) 0x7f28dc003520 con 0x7f28f806fc80 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f815a440 msgr2=0x7f28f815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f815a440 0x7f28f815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 >> v2:172.21.15.187:3300/0 conn(0x7f28f8072150 msgr2=0x7f28f80725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 --2- 172.21.15.150:0/563668397 >> v2:172.21.15.187:3300/0 conn(0x7f28f8072150 0x7f28f80725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f815d800 con 0x7f28f806fc80 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f28e0001690 con 0x7f28f806fc80 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 >> v1:172.21.15.150:6789/0 conn(0x7f28f806fc80 legacy=0x7f28f80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28dc0039b0 0x7f28dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f815a440 0x7f28dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f28f8072b00 con 0x7f28dc004800 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28f806e280 con 0x7f28dc0039b0 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28dc00b9c0 con 0x7f28f815a440 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28efe00640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f6800640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f815a440 0x7f28dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.954+0000 7f28f5e00640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28dc0039b0 0x7f28dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.955+0000 7f28efe00640 1 -- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f815a440 msgr2=0x7f28dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.955+0000 7f28efe00640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f815a440 0x7f28dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.955+0000 7f28efe00640 1 -- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28dc0039b0 msgr2=0x7f28dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:06.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.955+0000 7f28efe00640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28dc0039b0 0x7f28dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.955+0000 7f28efe00640 1 -- 172.21.15.150:0/563668397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc003520 con 0x7f28dc004800 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.956+0000 7f28efe00640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28dc004c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c210 tx=0x7f28dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2358cf46a9bda56f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.956+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0007d60 con 0x7f28dc004800 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.956+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e0030070 con 0x7f28dc004800 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.956+0000 7f28f5400640 1 -- 172.21.15.150:0/563668397 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e002cd20 con 0x7f28dc004800 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.956+0000 7f28fcc00640 1 -- 172.21.15.150:0/563668397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 msgr2=0x7f28dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.956+0000 7f28fcc00640 1 --2- 172.21.15.150:0/563668397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28dc004c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c210 tx=0x7f28dc005830 comp rx=0 tx=0).stop 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.956+0000 7f28f6800640 1 -- 172.21.15.150:0/563668397 reap_dead start 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.957+0000 7f28fcc00640 1 -- 172.21.15.150:0/563668397 shutdown_connections 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.957+0000 7f28fcc00640 1 -- 172.21.15.150:0/563668397 >> 172.21.15.150:0/563668397 conn(0x7f28f80b2460 msgr2=0x7f28f80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.957+0000 7f28fcc00640 1 -- 172.21.15.150:0/563668397 shutdown_connections 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.957+0000 7f28fcc00640 1 -- 172.21.15.150:0/563668397 wait complete. 2024-04-15T15:24:06.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.958+0000 7f28fcc00640 1 Processor -- start 2024-04-15T15:24:06.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.959+0000 7f28fcc00640 1 -- start start 2024-04-15T15:24:06.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.959+0000 7f28fcc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28f8144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.959+0000 7f28fcc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f81f5270 0x7f28f81593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.959+0000 7f28fcc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f80b4090 0x7f28f80b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.959+0000 7f28fcc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f28f815a820 con 0x7f28dc004800 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.959+0000 7f28fcc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28f80cd7f0 con 0x7f28f81f5270 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.959+0000 7f28fcc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28f815ddc0 con 0x7f28f80b4090 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28f8144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28f8144610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51000/0 (socket says 172.21.15.150:51000) 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 -- 172.21.15.150:0/2609476660 learned_addr learned my addr 172.21.15.150:0/2609476660 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28efe00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f81f5270 0x7f28f81593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f6800640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f80b4090 0x7f28f80b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 -- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f80b4090 msgr2=0x7f28f80b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f80b4090 0x7f28f80b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 -- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f81f5270 msgr2=0x7f28f81593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f81f5270 0x7f28f81593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:06.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 -- 172.21.15.150:0/2609476660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f80b3900 con 0x7f28dc004800 2024-04-15T15:24:06.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f6800640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f80b4090 0x7f28f80b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:06.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28efe00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f81f5270 0x7f28f81593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:06.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.960+0000 7f28f5e00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28f8144610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28e8011570 tx=0x7f28e8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.961+0000 7f28ee000640 1 -- 172.21.15.150:0/2609476660 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8015930 con 0x7f28dc004800 2024-04-15T15:24:06.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.961+0000 7f28ee000640 1 -- 172.21.15.150:0/2609476660 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e8015ad0 con 0x7f28dc004800 2024-04-15T15:24:06.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.961+0000 7f28ee000640 1 -- 172.21.15.150:0/2609476660 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800f460 con 0x7f28dc004800 2024-04-15T15:24:06.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.961+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f80b5200 con 0x7f28dc004800 2024-04-15T15:24:06.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.961+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f80b3b70 con 0x7f28dc004800 2024-04-15T15:24:06.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.962+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f80b5880 con 0x7f28dc004800 2024-04-15T15:24:06.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.963+0000 7f28ee000640 1 -- 172.21.15.150:0/2609476660 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28e800f600 con 0x7f28dc004800 2024-04-15T15:24:06.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.964+0000 7f28ee000640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28d406c8f0 0x7f28d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:06.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.964+0000 7f28ee000640 1 -- 172.21.15.150:0/2609476660 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f28e808c380 con 0x7f28dc004800 2024-04-15T15:24:06.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.965+0000 7f28efe00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28d406c8f0 0x7f28d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:06.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.966+0000 7f28efe00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28d406c8f0 0x7f28d406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f28e0034000 tx=0x7f28e0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:06.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:06.969+0000 7f28ee000640 1 -- 172.21.15.150:0/2609476660 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28e8019030 con 0x7f28dc004800 2024-04-15T15:24:07.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.168+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28f80cf4c0 con 0x7f28dc004800 2024-04-15T15:24:07.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.169+0000 7f28ee000640 1 -- 172.21.15.150:0/2609476660 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f28e805a7a0 con 0x7f28dc004800 2024-04-15T15:24:07.170 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:07.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.171+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28d406c8f0 msgr2=0x7f28d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.171+0000 7f28fcc00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28d406c8f0 0x7f28d406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f28e0034000 tx=0x7f28e0007000 comp rx=0 tx=0).stop 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.171+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 msgr2=0x7f28f8144610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.171+0000 7f28fcc00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28f8144610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28e8011570 tx=0x7f28e8011a30 comp rx=0 tx=0).stop 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.172+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 shutdown_connections 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.172+0000 7f28fcc00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28d406c8f0 0x7f28d406edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.172+0000 7f28fcc00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28f80b4090 0x7f28f80b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.172+0000 7f28fcc00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28f81f5270 0x7f28f81593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.172+0000 7f28fcc00640 1 --2- 172.21.15.150:0/2609476660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc004800 0x7f28f8144610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.172+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 >> 172.21.15.150:0/2609476660 conn(0x7f28f80b2460 msgr2=0x7f28f80adcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:07.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.172+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 shutdown_connections 2024-04-15T15:24:07.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.172+0000 7f28fcc00640 1 -- 172.21.15.150:0/2609476660 wait complete. 2024-04-15T15:24:07.193 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:07.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.405+0000 7f1abd800640 1 Processor -- start 2024-04-15T15:24:07.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.405+0000 7f1abd800640 1 -- start start 2024-04-15T15:24:07.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.407+0000 7f1abd800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ab8162b80 0x7f1ab80b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.407+0000 7f1abd800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162f60 0x7f1ab806f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.407+0000 7f1abd800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab8072810 con 0x7f1ab80b4390 2024-04-15T15:24:07.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.407+0000 7f1abd800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab80725f0 con 0x7f1ab8162b80 2024-04-15T15:24:07.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.407+0000 7f1abd800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab80b49c0 con 0x7f1ab8162f60 2024-04-15T15:24:07.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab7400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1ab80b4390 0x7f1ab8071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42198/0 (socket says 172.21.15.150:42198) 2024-04-15T15:24:07.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab7400640 1 -- 172.21.15.150:0/3610436039 learned_addr learned my addr 172.21.15.150:0/3610436039 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:07.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab7e00640 1 --2- 172.21.15.150:0/3610436039 >> v2:172.21.15.150:3300/0 conn(0x7f1ab8162b80 0x7f1ab80b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1abce00640 1 --2- 172.21.15.150:0/3610436039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162f60 0x7f1ab806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab6a00640 1 -- 172.21.15.150:0/3610436039 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2607087694 0 0) 0x7f1ab8072810 con 0x7f1ab80b4390 2024-04-15T15:24:07.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab6a00640 1 -- 172.21.15.150:0/3610436039 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a9c003520 con 0x7f1ab80b4390 2024-04-15T15:24:07.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab7e00640 1 -- 172.21.15.150:0/3610436039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162f60 msgr2=0x7f1ab806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:07.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab7e00640 1 --2- 172.21.15.150:0/3610436039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162f60 0x7f1ab806f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab7e00640 1 -- 172.21.15.150:0/3610436039 >> v1:172.21.15.150:6789/0 conn(0x7f1ab80b4390 legacy=0x7f1ab8071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1ab7e00640 1 -- 172.21.15.150:0/3610436039 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab80b4ca0 con 0x7f1ab8162b80 2024-04-15T15:24:07.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.408+0000 7f1abce00640 1 --2- 172.21.15.150:0/3610436039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162f60 0x7f1ab806f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:07.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.410+0000 7f1ab7e00640 1 --2- 172.21.15.150:0/3610436039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162b80 0x7f1ab80b5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80097a0 tx=0x7f1aa802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54af114690736f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:07.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.410+0000 7f1ab6a00640 1 -- 172.21.15.150:0/3610436039 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8035020 con 0x7f1ab8162b80 2024-04-15T15:24:07.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.410+0000 7f1ab6a00640 1 -- 172.21.15.150:0/3610436039 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa8031830 con 0x7f1ab8162b80 2024-04-15T15:24:07.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.410+0000 7f1ab6a00640 1 -- 172.21.15.150:0/3610436039 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8031c00 con 0x7f1ab8162b80 2024-04-15T15:24:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.411+0000 7f1abd800640 1 -- 172.21.15.150:0/3610436039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162b80 msgr2=0x7f1ab80b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.411+0000 7f1abd800640 1 --2- 172.21.15.150:0/3610436039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162b80 0x7f1ab80b5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80097a0 tx=0x7f1aa802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.411+0000 7f1abd800640 1 -- 172.21.15.150:0/3610436039 shutdown_connections 2024-04-15T15:24:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.411+0000 7f1abd800640 1 --2- 172.21.15.150:0/3610436039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162f60 0x7f1ab806f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.411+0000 7f1abd800640 1 --2- 172.21.15.150:0/3610436039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162b80 0x7f1ab80b5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.411+0000 7f1abd800640 1 -- 172.21.15.150:0/3610436039 >> 172.21.15.150:0/3610436039 conn(0x7f1ab806b930 msgr2=0x7f1ab806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.411+0000 7f1abd800640 1 -- 172.21.15.150:0/3610436039 shutdown_connections 2024-04-15T15:24:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.411+0000 7f1abd800640 1 -- 172.21.15.150:0/3610436039 wait complete. 2024-04-15T15:24:07.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.413+0000 7f1abd800640 1 Processor -- start 2024-04-15T15:24:07.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.413+0000 7f1abd800640 1 -- start start 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abd800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ab80b4390 0x7f1ab815c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abd800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162b80 0x7f1ab8161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abd800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162f60 0x7f1ab8161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abd800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab806f750 con 0x7f1ab8162f60 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abd800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab80b5260 con 0x7f1ab80b4390 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abd800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab806f8d0 con 0x7f1ab8162b80 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162f60 0x7f1ab8161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162f60 0x7f1ab8161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51008/0 (socket says 172.21.15.150:51008) 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1abce00640 1 -- 172.21.15.150:0/3056877264 learned_addr learned my addr 172.21.15.150:0/3056877264 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1ab7400640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162b80 0x7f1ab8161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.414+0000 7f1ab7e00640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ab80b4390 0x7f1ab815c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1abce00640 1 -- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162b80 msgr2=0x7f1ab8161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1abce00640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162b80 0x7f1ab8161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1abce00640 1 -- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ab80b4390 msgr2=0x7f1ab815c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1abce00640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ab80b4390 0x7f1ab815c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1abce00640 1 -- 172.21.15.150:0/3056877264 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab81653f0 con 0x7f1ab8162f60 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1ab7400640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162b80 0x7f1ab8161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1abce00640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162f60 0x7f1ab8161e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009fa0 tx=0x7f1ab000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1aafe00640 1 -- 172.21.15.150:0/3056877264 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab001a070 con 0x7f1ab8162f60 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1aafe00640 1 -- 172.21.15.150:0/3056877264 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0003bb0 con 0x7f1ab8162f60 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1aafe00640 1 -- 172.21.15.150:0/3056877264 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0004030 con 0x7f1ab8162f60 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab8164630 con 0x7f1ab8162f60 2024-04-15T15:24:07.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.415+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab8164bf0 con 0x7f1ab8162f60 2024-04-15T15:24:07.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.416+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab806c8c0 con 0x7f1ab8162f60 2024-04-15T15:24:07.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.417+0000 7f1aafe00640 1 -- 172.21.15.150:0/3056877264 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ab001d070 con 0x7f1ab8162f60 2024-04-15T15:24:07.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.418+0000 7f1aafe00640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.418+0000 7f1aafe00640 1 -- 172.21.15.150:0/3056877264 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1ab008bf00 con 0x7f1ab8162f60 2024-04-15T15:24:07.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.419+0000 7f1ab7e00640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.420+0000 7f1ab7e00640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1a9406c8f0 0x7f1a9406edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8036000 tx=0x7f1aa8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:07.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.423+0000 7f1aafe00640 1 -- 172.21.15.150:0/3056877264 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ab0010070 con 0x7f1ab8162f60 2024-04-15T15:24:07.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.621+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ab80721c0 con 0x7f1ab8162f60 2024-04-15T15:24:07.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.622+0000 7f1aafe00640 1 -- 172.21.15.150:0/3056877264 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f1ab005a290 con 0x7f1ab8162f60 2024-04-15T15:24:07.624 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:07.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1a9406c8f0 msgr2=0x7f1a9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1a9406c8f0 0x7f1a9406edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8036000 tx=0x7f1aa8008030 comp rx=0 tx=0).stop 2024-04-15T15:24:07.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162f60 msgr2=0x7f1ab8161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162f60 0x7f1ab8161e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009fa0 tx=0x7f1ab000d560 comp rx=0 tx=0).stop 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 shutdown_connections 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ab8162f60 0x7f1ab8161e50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ab8162b80 0x7f1ab8161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 --2- 172.21.15.150:0/3056877264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ab80b4390 0x7f1ab815c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.624+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 >> 172.21.15.150:0/3056877264 conn(0x7f1ab806b930 msgr2=0x7f1ab80cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.625+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 shutdown_connections 2024-04-15T15:24:07.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.625+0000 7f1abd800640 1 -- 172.21.15.150:0/3056877264 wait complete. 2024-04-15T15:24:07.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:07.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.858+0000 7f5159000640 1 Processor -- start 2024-04-15T15:24:07.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.858+0000 7f5159000640 1 -- start start 2024-04-15T15:24:07.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.860+0000 7f5159000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 0x7f51540b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.860+0000 7f5159000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5154162dd0 con 0x7f515415c680 2024-04-15T15:24:07.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.860+0000 7f5159000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51541630b0 con 0x7f5154162730 2024-04-15T15:24:07.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.860+0000 7f5159000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5154162b90 con 0x7f51540d3260 2024-04-15T15:24:07.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5153400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f515415c680 0x7f515415f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42212/0 (socket says 172.21.15.150:42212) 2024-04-15T15:24:07.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5153400640 1 -- 172.21.15.150:0/2193850008 learned_addr learned my addr 172.21.15.150:0/2193850008 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:07.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5153e00640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 0x7f51540b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984225667 0 0) 0x7f5154162dd0 con 0x7f515415c680 2024-04-15T15:24:07.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5138003520 con 0x7f515415c680 2024-04-15T15:24:07.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 243191211 0 0) 0x7f5138003520 con 0x7f515415c680 2024-04-15T15:24:07.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 msgr2=0x7f51540b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:07.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 0x7f51540b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 >> v1:172.21.15.187:6789/0 conn(0x7f5154162730 legacy=0x7f51540d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5154163390 con 0x7f515415c680 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f513c001670 con 0x7f515415c680 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5153e00640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 0x7f51540b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 >> v1:172.21.15.150:6789/0 conn(0x7f515415c680 legacy=0x7f515415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.861+0000 7f5152000640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51380039e0 0x7f5138003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5152000640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5138004800 0x7f5138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5152000640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 0x7f51380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5154162b90 con 0x7f51380039e0 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f513800ba70 con 0x7f5138004800 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f513800bc40 con 0x7f51540d3260 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5153400640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51380039e0 0x7f5138003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5153e00640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 0x7f51380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5153400640 1 -- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 msgr2=0x7f51380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:07.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5153400640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540d3260 0x7f51380051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5153400640 1 -- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5138004800 msgr2=0x7f5138004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:07.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5152a00640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5138004800 0x7f5138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5153400640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5138004800 0x7f5138004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5153400640 1 -- 172.21.15.150:0/2193850008 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5138003520 con 0x7f51380039e0 2024-04-15T15:24:07.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.862+0000 7f5152a00640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5138004800 0x7f5138004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.863+0000 7f5153400640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51380039e0 0x7f5138003e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f513c007fd0 tx=0x7f5138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc96f853c4f0f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c007d50 con 0x7f51380039e0 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513c030070 con 0x7f51380039e0 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5152000640 1 -- 172.21.15.150:0/2193850008 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c005d30 con 0x7f51380039e0 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5159000640 1 -- 172.21.15.150:0/2193850008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51380039e0 msgr2=0x7f5138003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5159000640 1 --2- 172.21.15.150:0/2193850008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51380039e0 0x7f5138003e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f513c007fd0 tx=0x7f5138005830 comp rx=0 tx=0).stop 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5153e00640 1 -- 172.21.15.150:0/2193850008 reap_dead start 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5159000640 1 -- 172.21.15.150:0/2193850008 shutdown_connections 2024-04-15T15:24:07.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5159000640 1 -- 172.21.15.150:0/2193850008 >> 172.21.15.150:0/2193850008 conn(0x7f5154073480 msgr2=0x7f5154073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:07.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5159000640 1 -- 172.21.15.150:0/2193850008 shutdown_connections 2024-04-15T15:24:07.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.864+0000 7f5159000640 1 -- 172.21.15.150:0/2193850008 wait complete. 2024-04-15T15:24:07.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.866+0000 7f5159000640 1 Processor -- start 2024-04-15T15:24:07.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.866+0000 7f5159000640 1 -- start start 2024-04-15T15:24:07.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.867+0000 7f5159000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51380039e0 0x7f5154158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.867+0000 7f5159000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51540b9040 0x7f51540c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.867+0000 7f5159000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540c9cb0 0x7f515415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.867+0000 7f5159000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f515415ff60 con 0x7f51540b9040 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.867+0000 7f5159000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f515415ca60 con 0x7f51380039e0 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.867+0000 7f5159000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51540b6c70 con 0x7f51540c9cb0 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.867+0000 7f5152a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51540b9040 0x7f51540c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5152a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51540b9040 0x7f51540c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51028/0 (socket says 172.21.15.150:51028) 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5152a00640 1 -- 172.21.15.150:0/1355503668 learned_addr learned my addr 172.21.15.150:0/1355503668 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5153400640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51380039e0 0x7f5154158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5153e00640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540c9cb0 0x7f515415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5152a00640 1 -- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540c9cb0 msgr2=0x7f515415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5152a00640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540c9cb0 0x7f515415a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5152a00640 1 -- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51380039e0 msgr2=0x7f5154158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:07.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5152a00640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51380039e0 0x7f5154158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:07.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5152a00640 1 -- 172.21.15.150:0/1355503668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51540c8c10 con 0x7f51540b9040 2024-04-15T15:24:07.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f5152a00640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51540b9040 0x7f51540c8310 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f514400dae0 tx=0x7f5144002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:07.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.868+0000 7f514b400640 1 -- 172.21.15.150:0/1355503668 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144003da0 con 0x7f51540b9040 2024-04-15T15:24:07.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.869+0000 7f514b400640 1 -- 172.21.15.150:0/1355503668 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5144017070 con 0x7f51540b9040 2024-04-15T15:24:07.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.869+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51540c8ef0 con 0x7f51540b9040 2024-04-15T15:24:07.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.869+0000 7f514b400640 1 -- 172.21.15.150:0/1355503668 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514400b950 con 0x7f51540b9040 2024-04-15T15:24:07.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.869+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51540c94b0 con 0x7f51540b9040 2024-04-15T15:24:07.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.870+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51540755c0 con 0x7f51540b9040 2024-04-15T15:24:07.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.871+0000 7f514b400640 1 -- 172.21.15.150:0/1355503668 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f514400baf0 con 0x7f51540b9040 2024-04-15T15:24:07.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.872+0000 7f514b400640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f513006c8f0 0x7f513006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:07.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.872+0000 7f514b400640 1 -- 172.21.15.150:0/1355503668 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f514408bd10 con 0x7f51540b9040 2024-04-15T15:24:07.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.872+0000 7f5153400640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f513006c8f0 0x7f513006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:07.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.873+0000 7f5153400640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f513006c8f0 0x7f513006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f513c034000 tx=0x7f513c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:07.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:07.876+0000 7f514b400640 1 -- 172.21.15.150:0/1355503668 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f514405a140 con 0x7f51540b9040 2024-04-15T15:24:08.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.074+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f515406dbe0 con 0x7f51540b9040 2024-04-15T15:24:08.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.075+0000 7f514b400640 1 -- 172.21.15.150:0/1355503668 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f5144059a40 con 0x7f51540b9040 2024-04-15T15:24:08.077 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:08.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f513006c8f0 msgr2=0x7f513006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f513006c8f0 0x7f513006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f513c034000 tx=0x7f513c009030 comp rx=0 tx=0).stop 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51540b9040 msgr2=0x7f51540c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51540b9040 0x7f51540c8310 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f514400dae0 tx=0x7f5144002070 comp rx=0 tx=0).stop 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 shutdown_connections 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f513006c8f0 0x7f513006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51540c9cb0 0x7f515415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51540b9040 0x7f51540c8310 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 --2- 172.21.15.150:0/1355503668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51380039e0 0x7f5154158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 >> 172.21.15.150:0/1355503668 conn(0x7f5154073480 msgr2=0x7f51540d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 shutdown_connections 2024-04-15T15:24:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.078+0000 7f5159000640 1 -- 172.21.15.150:0/1355503668 wait complete. 2024-04-15T15:24:08.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:24:08.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.310+0000 7fd1e5a00640 1 Processor -- start 2024-04-15T15:24:08.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.310+0000 7fd1e5a00640 1 -- start start 2024-04-15T15:24:08.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.311+0000 7fd1e5a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd1e015b6d0 0x7fd1e00cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.311+0000 7fd1e5a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e00b51f0 0x7fd1e015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.311+0000 7fd1e5a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e006f940 con 0x7fd1e0070210 2024-04-15T15:24:08.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.311+0000 7fd1e5a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e006e8a0 con 0x7fd1e015b6d0 2024-04-15T15:24:08.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.311+0000 7fd1e5a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e006eb80 con 0x7fd1e00b51f0 2024-04-15T15:24:08.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1dea00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd1e015b6d0 0x7fd1e00cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1dea00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd1e015b6d0 0x7fd1e00cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51040/0 (socket says 172.21.15.150:51040) 2024-04-15T15:24:08.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1dea00640 1 -- 172.21.15.150:0/2976307291 learned_addr learned my addr 172.21.15.150:0/2976307291 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:08.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1df400640 1 --2- 172.21.15.150:0/2976307291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e00b51f0 0x7fd1e015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1dea00640 1 -- 172.21.15.150:0/2976307291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e00b51f0 msgr2=0x7fd1e015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:08.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1dea00640 1 --2- 172.21.15.150:0/2976307291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e00b51f0 0x7fd1e015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1dea00640 1 -- 172.21.15.150:0/2976307291 >> v1:172.21.15.187:6789/0 conn(0x7fd1e0070210 legacy=0x7fd1e00b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:08.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1dea00640 1 -- 172.21.15.150:0/2976307291 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e006ee60 con 0x7fd1e015b6d0 2024-04-15T15:24:08.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.312+0000 7fd1df400640 1 --2- 172.21.15.150:0/2976307291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e00b51f0 0x7fd1e015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:08.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.313+0000 7fd1dea00640 1 --2- 172.21.15.150:0/2976307291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e00cdd10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000bb10 tx=0x7fd1d002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de9368a5c5500a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:08.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.314+0000 7fd1de000640 1 -- 172.21.15.150:0/2976307291 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d003b070 con 0x7fd1e015b6d0 2024-04-15T15:24:08.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.314+0000 7fd1de000640 1 -- 172.21.15.150:0/2976307291 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d002ede0 con 0x7fd1e015b6d0 2024-04-15T15:24:08.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.314+0000 7fd1de000640 1 -- 172.21.15.150:0/2976307291 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0036660 con 0x7fd1e015b6d0 2024-04-15T15:24:08.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.314+0000 7fd1e5a00640 1 -- 172.21.15.150:0/2976307291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 msgr2=0x7fd1e00cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.314+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/2976307291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e00cdd10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000bb10 tx=0x7fd1d002e030 comp rx=0 tx=0).stop 2024-04-15T15:24:08.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.314+0000 7fd1e5a00640 1 -- 172.21.15.150:0/2976307291 shutdown_connections 2024-04-15T15:24:08.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.314+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/2976307291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e00b51f0 0x7fd1e015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.315+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/2976307291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e00cdd10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.315+0000 7fd1e5a00640 1 -- 172.21.15.150:0/2976307291 >> 172.21.15.150:0/2976307291 conn(0x7fd1e00b27b0 msgr2=0x7fd1e00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:08.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.315+0000 7fd1e5a00640 1 -- 172.21.15.150:0/2976307291 shutdown_connections 2024-04-15T15:24:08.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.315+0000 7fd1e5a00640 1 -- 172.21.15.150:0/2976307291 wait complete. 2024-04-15T15:24:08.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.316+0000 7fd1e5a00640 1 Processor -- start 2024-04-15T15:24:08.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.316+0000 7fd1e5a00640 1 -- start start 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.317+0000 7fd1e5a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e0070210 0x7fd1e00c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.317+0000 7fd1e5a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1e00b51f0 0x7fd1e01556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.317+0000 7fd1e5a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e01f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.317+0000 7fd1e5a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e00ce250 con 0x7fd1e015b6d0 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.317+0000 7fd1e5a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e006f420 con 0x7fd1e00b51f0 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.317+0000 7fd1e5a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e00ce3d0 con 0x7fd1e0070210 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1df400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e01f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1df400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e01f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51042/0 (socket says 172.21.15.150:51042) 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1df400640 1 -- 172.21.15.150:0/3221692877 learned_addr learned my addr 172.21.15.150:0/3221692877 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1dea00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e0070210 0x7fd1e00c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1d7e00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1e00b51f0 0x7fd1e01556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1df400640 1 -- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e0070210 msgr2=0x7fd1e00c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1df400640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e0070210 0x7fd1e00c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1df400640 1 -- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1e00b51f0 msgr2=0x7fd1e01556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1df400640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1e00b51f0 0x7fd1e01556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.318+0000 7fd1df400640 1 -- 172.21.15.150:0/3221692877 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e0155c20 con 0x7fd1e015b6d0 2024-04-15T15:24:08.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.319+0000 7fd1df400640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e01f75d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800ce80 tx=0x7fd1d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:08.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.319+0000 7fd1d6a00640 1 -- 172.21.15.150:0/3221692877 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d801a070 con 0x7fd1e015b6d0 2024-04-15T15:24:08.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.319+0000 7fd1d6a00640 1 -- 172.21.15.150:0/3221692877 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d800d880 con 0x7fd1e015b6d0 2024-04-15T15:24:08.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.319+0000 7fd1d6a00640 1 -- 172.21.15.150:0/3221692877 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d800dc70 con 0x7fd1e015b6d0 2024-04-15T15:24:08.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.319+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e01f7b10 con 0x7fd1e015b6d0 2024-04-15T15:24:08.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.319+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e00c6100 con 0x7fd1e015b6d0 2024-04-15T15:24:08.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.320+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e0068c80 con 0x7fd1e015b6d0 2024-04-15T15:24:08.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.321+0000 7fd1d6a00640 1 -- 172.21.15.150:0/3221692877 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1d801d070 con 0x7fd1e015b6d0 2024-04-15T15:24:08.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.322+0000 7fd1d6a00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd1bc06c8f0 0x7fd1bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.322+0000 7fd1d6a00640 1 -- 172.21.15.150:0/3221692877 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd1d808c300 con 0x7fd1e015b6d0 2024-04-15T15:24:08.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.323+0000 7fd1dea00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd1bc06c8f0 0x7fd1bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.324+0000 7fd1dea00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd1bc06c8f0 0x7fd1bc06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0038000 tx=0x7fd1d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:08.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.327+0000 7fd1d6a00640 1 -- 172.21.15.150:0/3221692877 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1d805a040 con 0x7fd1e015b6d0 2024-04-15T15:24:08.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.568+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1e00d0560 con 0x7fd1e015b6d0 2024-04-15T15:24:08.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.569+0000 7fd1d6a00640 1 -- 172.21.15.150:0/3221692877 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7fd1d805e340 con 0x7fd1e015b6d0 2024-04-15T15:24:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:08.571 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:23:54.683053+0000","modified":"2024-04-15T15:23:57.779875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4250,"qdb_cluster":[4250]},"id":1}]} 2024-04-15T15:24:08.571 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-04-15T15:24:08.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd1bc06c8f0 msgr2=0x7fd1bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd1bc06c8f0 0x7fd1bc06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0038000 tx=0x7fd1d0008040 comp rx=0 tx=0).stop 2024-04-15T15:24:08.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 msgr2=0x7fd1e01f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e01f75d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800ce80 tx=0x7fd1d800a710 comp rx=0 tx=0).stop 2024-04-15T15:24:08.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 shutdown_connections 2024-04-15T15:24:08.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e015b6d0 0x7fd1e01f75d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1e00b51f0 0x7fd1e01556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd1bc06c8f0 0x7fd1bc06edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 --2- 172.21.15.150:0/3221692877 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1e0070210 0x7fd1e00c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 >> 172.21.15.150:0/3221692877 conn(0x7fd1e00b27b0 msgr2=0x7fd1e00b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:08.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 shutdown_connections 2024-04-15T15:24:08.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.572+0000 7fd1e5a00640 1 -- 172.21.15.150:0/3221692877 wait complete. 2024-04-15T15:24:08.594 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:08.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.799+0000 7f612fe00640 1 Processor -- start 2024-04-15T15:24:08.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.799+0000 7f612fe00640 1 -- start start 2024-04-15T15:24:08.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.800+0000 7f612fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f61300cee00 0x7f613006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.800+0000 7f612fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6130071f50 0x7f61300723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.801+0000 7f612fe00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f613015dbb0 con 0x7f613015cbe0 2024-04-15T15:24:08.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.801+0000 7f612fe00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f613006f4f0 con 0x7f6130071f50 2024-04-15T15:24:08.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.801+0000 7f612fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f61300728f0 con 0x7f61300cee00 2024-04-15T15:24:08.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.801+0000 7f612f400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f613015cbe0 0x7f613015d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42228/0 (socket says 172.21.15.150:42228) 2024-04-15T15:24:08.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.801+0000 7f612f400640 1 -- 172.21.15.150:0/187271808 learned_addr learned my addr 172.21.15.150:0/187271808 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:08.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.801+0000 7f6126000640 1 --2- 172.21.15.150:0/187271808 >> v2:172.21.15.150:3300/0 conn(0x7f6130071f50 0x7f61300723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.801+0000 7f612ea00640 1 --2- 172.21.15.150:0/187271808 >> v2:172.21.15.187:3300/0 conn(0x7f61300cee00 0x7f613006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.801+0000 7f612e000640 1 -- 172.21.15.150:0/187271808 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635535955 0 0) 0x7f613015dbb0 con 0x7f613015cbe0 2024-04-15T15:24:08.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.802+0000 7f612e000640 1 -- 172.21.15.150:0/187271808 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6114003520 con 0x7f613015cbe0 2024-04-15T15:24:08.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.802+0000 7f6126000640 1 -- 172.21.15.150:0/187271808 >> v2:172.21.15.187:3300/0 conn(0x7f61300cee00 msgr2=0x7f613006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:08.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.802+0000 7f6126000640 1 --2- 172.21.15.150:0/187271808 >> v2:172.21.15.187:3300/0 conn(0x7f61300cee00 0x7f613006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.802+0000 7f6126000640 1 -- 172.21.15.150:0/187271808 >> v1:172.21.15.150:6789/0 conn(0x7f613015cbe0 legacy=0x7f613015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.802+0000 7f6126000640 1 -- 172.21.15.150:0/187271808 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613015d8d0 con 0x7f6130071f50 2024-04-15T15:24:08.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.803+0000 7f6126000640 1 --2- 172.21.15.150:0/187271808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6130071f50 0x7f61300723b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6130070360 tx=0x7f611802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca73c733446a0de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:08.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.803+0000 7f612e000640 1 -- 172.21.15.150:0/187271808 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118035020 con 0x7f6130071f50 2024-04-15T15:24:08.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.803+0000 7f612e000640 1 -- 172.21.15.150:0/187271808 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6118031830 con 0x7f6130071f50 2024-04-15T15:24:08.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.803+0000 7f612e000640 1 -- 172.21.15.150:0/187271808 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118031c00 con 0x7f6130071f50 2024-04-15T15:24:08.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.804+0000 7f612fe00640 1 -- 172.21.15.150:0/187271808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6130071f50 msgr2=0x7f61300723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.804+0000 7f612fe00640 1 --2- 172.21.15.150:0/187271808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6130071f50 0x7f61300723b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6130070360 tx=0x7f611802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.804+0000 7f612fe00640 1 -- 172.21.15.150:0/187271808 shutdown_connections 2024-04-15T15:24:08.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.804+0000 7f612fe00640 1 --2- 172.21.15.150:0/187271808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6130071f50 0x7f61300723b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.804+0000 7f612fe00640 1 --2- 172.21.15.150:0/187271808 >> v2:172.21.15.187:3300/0 conn(0x7f61300cee00 0x7f613006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.804+0000 7f612fe00640 1 -- 172.21.15.150:0/187271808 >> 172.21.15.150:0/187271808 conn(0x7f613006b930 msgr2=0x7f613006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:08.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.804+0000 7f612fe00640 1 -- 172.21.15.150:0/187271808 shutdown_connections 2024-04-15T15:24:08.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.804+0000 7f612fe00640 1 -- 172.21.15.150:0/187271808 wait complete. 2024-04-15T15:24:08.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.806+0000 7f612fe00640 1 Processor -- start 2024-04-15T15:24:08.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.806+0000 7f612fe00640 1 -- start start 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6130071f50 0x7f61300a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61300cee00 0x7f6130148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f613015cbe0 0x7f61300accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f613006e3a0 con 0x7f613015cbe0 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f613006e0c0 con 0x7f61300cee00 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f613015dda0 con 0x7f6130071f50 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f613015cbe0 0x7f61300accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f613015cbe0 0x7f61300accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51052/0 (socket says 172.21.15.150:51052) 2024-04-15T15:24:08.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.807+0000 7f612f400640 1 -- 172.21.15.150:0/2766505414 learned_addr learned my addr 172.21.15.150:0/2766505414 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f612ea00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6130071f50 0x7f61300a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f6126000640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61300cee00 0x7f6130148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f612f400640 1 -- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6130071f50 msgr2=0x7f61300a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f612f400640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6130071f50 0x7f61300a9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f612f400640 1 -- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61300cee00 msgr2=0x7f6130148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f612f400640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61300cee00 0x7f6130148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f612f400640 1 -- 172.21.15.150:0/2766505414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61300aa890 con 0x7f613015cbe0 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f612f400640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f613015cbe0 0x7f61300accd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f612800c390 tx=0x7f612800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f6127400640 1 -- 172.21.15.150:0/2766505414 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128017070 con 0x7f613015cbe0 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f6127400640 1 -- 172.21.15.150:0/2766505414 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6128005ce0 con 0x7f613015cbe0 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61300aae50 con 0x7f613015cbe0 2024-04-15T15:24:08.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.808+0000 7f6127400640 1 -- 172.21.15.150:0/2766505414 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128007310 con 0x7f613015cbe0 2024-04-15T15:24:08.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.809+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61300ab410 con 0x7f613015cbe0 2024-04-15T15:24:08.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.810+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61300755c0 con 0x7f613015cbe0 2024-04-15T15:24:08.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.811+0000 7f6127400640 1 -- 172.21.15.150:0/2766505414 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f612801a070 con 0x7f613015cbe0 2024-04-15T15:24:08.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.812+0000 7f6127400640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f610c06c920 0x7f610c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:08.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.812+0000 7f6127400640 1 -- 172.21.15.150:0/2766505414 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f612808af20 con 0x7f613015cbe0 2024-04-15T15:24:08.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.812+0000 7f612ea00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f610c06c920 0x7f610c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:08.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.813+0000 7f612ea00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f610c06c920 0x7f610c06ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6120009f30 tx=0x7f612000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:08.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:08.816+0000 7f6127400640 1 -- 172.21.15.150:0/2766505414 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6128059340 con 0x7f613015cbe0 2024-04-15T15:24:09.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.016+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6130159e50 con 0x7f613015cbe0 2024-04-15T15:24:09.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.019+0000 7f6127400640 1 -- 172.21.15.150:0/2766505414 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7f6128058c40 con 0x7f613015cbe0 2024-04-15T15:24:09.021 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:09.021 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:09.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f610c06c920 msgr2=0x7f610c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f610c06c920 0x7f610c06ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6120009f30 tx=0x7f612000b040 comp rx=0 tx=0).stop 2024-04-15T15:24:09.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f613015cbe0 msgr2=0x7f61300accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f613015cbe0 0x7f61300accd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f612800c390 tx=0x7f612800c850 comp rx=0 tx=0).stop 2024-04-15T15:24:09.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 shutdown_connections 2024-04-15T15:24:09.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f613015cbe0 0x7f61300accd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61300cee00 0x7f6130148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f610c06c920 0x7f610c06ede0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 --2- 172.21.15.150:0/2766505414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6130071f50 0x7f61300a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 >> 172.21.15.150:0/2766505414 conn(0x7f613006b930 msgr2=0x7f61300b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:09.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 shutdown_connections 2024-04-15T15:24:09.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.022+0000 7f612fe00640 1 -- 172.21.15.150:0/2766505414 wait complete. 2024-04-15T15:24:09.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:09.044 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:09.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.263+0000 7fcdacc00640 1 Processor -- start 2024-04-15T15:24:09.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.263+0000 7fcdacc00640 1 -- start start 2024-04-15T15:24:09.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.264+0000 7fcdacc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcda815ecc0 0x7fcda80d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.264+0000 7fcdacc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcda80b3720 0x7fcda80cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.264+0000 7fcdacc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcda80ba4d0 con 0x7fcda80d3270 2024-04-15T15:24:09.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.264+0000 7fcdacc00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fcda815f0a0 con 0x7fcda815ecc0 2024-04-15T15:24:09.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.264+0000 7fcdacc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcda80ba280 con 0x7fcda80b3720 2024-04-15T15:24:09.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.265+0000 7fcda5e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcda815ecc0 0x7fcda80d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.265+0000 7fcda5e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcda815ecc0 0x7fcda80d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51060/0 (socket says 172.21.15.150:51060) 2024-04-15T15:24:09.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.265+0000 7fcda5e00640 1 -- 172.21.15.150:0/894360635 learned_addr learned my addr 172.21.15.150:0/894360635 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:09.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.265+0000 7fcda5e00640 1 -- 172.21.15.150:0/894360635 >> v2:172.21.15.187:3300/0 conn(0x7fcda80b3720 msgr2=0x7fcda80cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:09.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.265+0000 7fcda6800640 1 --2- 172.21.15.150:0/894360635 >> v2:172.21.15.187:3300/0 conn(0x7fcda80b3720 0x7fcda80cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.265+0000 7fcda5e00640 1 --2- 172.21.15.150:0/894360635 >> v2:172.21.15.187:3300/0 conn(0x7fcda80b3720 0x7fcda80cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.265+0000 7fcda5e00640 1 -- 172.21.15.150:0/894360635 >> v1:172.21.15.187:6789/0 conn(0x7fcda80d3270 legacy=0x7fcda80b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:09.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.265+0000 7fcda5e00640 1 -- 172.21.15.150:0/894360635 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda80ba7b0 con 0x7fcda815ecc0 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.266+0000 7fcda5e00640 1 --2- 172.21.15.150:0/894360635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda815ecc0 0x7fcda80d2d30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcda80d5970 tx=0x7fcd9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f669c94f5d0c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcda5400640 1 -- 172.21.15.150:0/894360635 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98035020 con 0x7fcda815ecc0 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcda5400640 1 -- 172.21.15.150:0/894360635 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd98031870 con 0x7fcda815ecc0 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcda5400640 1 -- 172.21.15.150:0/894360635 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98031c40 con 0x7fcda815ecc0 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcdacc00640 1 -- 172.21.15.150:0/894360635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda815ecc0 msgr2=0x7fcda80d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcdacc00640 1 --2- 172.21.15.150:0/894360635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda815ecc0 0x7fcda80d2d30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcda80d5970 tx=0x7fcd9802b910 comp rx=0 tx=0).stop 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcdacc00640 1 -- 172.21.15.150:0/894360635 shutdown_connections 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcdacc00640 1 --2- 172.21.15.150:0/894360635 >> v2:172.21.15.187:3300/0 conn(0x7fcda80b3720 0x7fcda80cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcdacc00640 1 --2- 172.21.15.150:0/894360635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda815ecc0 0x7fcda80d2d30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.267+0000 7fcdacc00640 1 -- 172.21.15.150:0/894360635 >> 172.21.15.150:0/894360635 conn(0x7fcda8073480 msgr2=0x7fcda8073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:09.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.268+0000 7fcdacc00640 1 -- 172.21.15.150:0/894360635 shutdown_connections 2024-04-15T15:24:09.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.268+0000 7fcdacc00640 1 -- 172.21.15.150:0/894360635 wait complete. 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.269+0000 7fcdacc00640 1 Processor -- start 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.269+0000 7fcdacc00640 1 -- start start 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.270+0000 7fcdacc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda80b3720 0x7fcda80ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.270+0000 7fcdacc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda80d3270 0x7fcda81524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.270+0000 7fcdacc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda815ecc0 0x7fcda8161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.270+0000 7fcdacc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcda80b8aa0 con 0x7fcda80b3720 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.270+0000 7fcdacc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcda80b90d0 con 0x7fcda80d3270 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.270+0000 7fcdacc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcda80b9d00 con 0x7fcda815ecc0 2024-04-15T15:24:09.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.270+0000 7fcda5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda80b3720 0x7fcda80ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda80b3720 0x7fcda80ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51062/0 (socket says 172.21.15.150:51062) 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda5e00640 1 -- 172.21.15.150:0/4218169597 learned_addr learned my addr 172.21.15.150:0/4218169597 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda6800640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda815ecc0 0x7fcda8161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcd9d400640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda80d3270 0x7fcda81524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda5e00640 1 -- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda815ecc0 msgr2=0x7fcda8161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda5e00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda815ecc0 0x7fcda8161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda5e00640 1 -- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda80d3270 msgr2=0x7fcda81524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda5e00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda80d3270 0x7fcda81524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda5e00640 1 -- 172.21.15.150:0/4218169597 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda80c9090 con 0x7fcda80b3720 2024-04-15T15:24:09.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda6800640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda815ecc0 0x7fcda8161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:09.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.271+0000 7fcda5e00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda80b3720 0x7fcda80ca130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcd9802b950 tx=0x7fcd980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:09.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.272+0000 7fcd9ea00640 1 -- 172.21.15.150:0/4218169597 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98040070 con 0x7fcda80b3720 2024-04-15T15:24:09.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.272+0000 7fcd9ea00640 1 -- 172.21.15.150:0/4218169597 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd9803a420 con 0x7fcda80b3720 2024-04-15T15:24:09.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.272+0000 7fcd9ea00640 1 -- 172.21.15.150:0/4218169597 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98035040 con 0x7fcda80b3720 2024-04-15T15:24:09.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.272+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda80c9370 con 0x7fcda80b3720 2024-04-15T15:24:09.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.272+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda80c9930 con 0x7fcda80b3720 2024-04-15T15:24:09.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.273+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda80d2cb0 con 0x7fcda80b3720 2024-04-15T15:24:09.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.274+0000 7fcd9ea00640 1 -- 172.21.15.150:0/4218169597 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd98038020 con 0x7fcda80b3720 2024-04-15T15:24:09.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.275+0000 7fcd9ea00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcd8406c8f0 0x7fcd8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.275+0000 7fcd9ea00640 1 -- 172.21.15.150:0/4218169597 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcd980adbe0 con 0x7fcda80b3720 2024-04-15T15:24:09.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.275+0000 7fcd9d400640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcd8406c8f0 0x7fcd8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.276+0000 7fcd9d400640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcd8406c8f0 0x7fcd8406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcda8073a60 tx=0x7fcd90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:09.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.279+0000 7fcd9ea00640 1 -- 172.21.15.150:0/4218169597 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd9807bf70 con 0x7fcda80b3720 2024-04-15T15:24:09.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.482+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcda8068c80 con 0x7fcda80b3720 2024-04-15T15:24:09.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.483+0000 7fcd9ea00640 1 -- 172.21.15.150:0/4218169597 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fcd9807b870 con 0x7fcda80b3720 2024-04-15T15:24:09.485 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcd8406c8f0 msgr2=0x7fcd8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcd8406c8f0 0x7fcd8406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcda8073a60 tx=0x7fcd90009040 comp rx=0 tx=0).stop 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda80b3720 msgr2=0x7fcda80ca130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda80b3720 0x7fcda80ca130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcd9802b950 tx=0x7fcd980046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 shutdown_connections 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcd8406c8f0 0x7fcd8406edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcda815ecc0 0x7fcda8161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcda80d3270 0x7fcda81524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 --2- 172.21.15.150:0/4218169597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda80b3720 0x7fcda80ca130 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 >> 172.21.15.150:0/4218169597 conn(0x7fcda8073480 msgr2=0x7fcda80b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 shutdown_connections 2024-04-15T15:24:09.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.486+0000 7fcdacc00640 1 -- 172.21.15.150:0/4218169597 wait complete. 2024-04-15T15:24:09.507 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:09.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.712+0000 7fda12e00640 1 Processor -- start 2024-04-15T15:24:09.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.712+0000 7fda12e00640 1 -- start start 2024-04-15T15:24:09.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.714+0000 7fda12e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fda0c0ce650 0x7fda0c09ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.714+0000 7fda12e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fda0c0d17c0 0x7fda0c09b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.714+0000 7fda12e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0c071330 con 0x7fda0c0703d0 2024-04-15T15:24:09.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.714+0000 7fda12e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fda0c09bb40 con 0x7fda0c0ce650 2024-04-15T15:24:09.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.714+0000 7fda12e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fda0c06ce50 con 0x7fda0c0d17c0 2024-04-15T15:24:09.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.714+0000 7fda0b400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fda0c0703d0 0x7fda0c06d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42240/0 (socket says 172.21.15.150:42240) 2024-04-15T15:24:09.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.714+0000 7fda0b400640 1 -- 172.21.15.150:0/555571902 learned_addr learned my addr 172.21.15.150:0/555571902 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:09.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.714+0000 7fda0be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fda0c0ce650 0x7fda0c09ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.715+0000 7fda10e00640 1 --2- 172.21.15.150:0/555571902 >> v2:172.21.15.187:3300/0 conn(0x7fda0c0d17c0 0x7fda0c09b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.715+0000 7fda0aa00640 1 -- 172.21.15.150:0/555571902 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974459030 0 0) 0x7fda0c071330 con 0x7fda0c0703d0 2024-04-15T15:24:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.715+0000 7fda0aa00640 1 -- 172.21.15.150:0/555571902 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9f0003520 con 0x7fda0c0703d0 2024-04-15T15:24:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.715+0000 7fda0be00640 1 -- 172.21.15.150:0/555571902 >> v2:172.21.15.187:3300/0 conn(0x7fda0c0d17c0 msgr2=0x7fda0c09b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.715+0000 7fda0be00640 1 --2- 172.21.15.150:0/555571902 >> v2:172.21.15.187:3300/0 conn(0x7fda0c0d17c0 0x7fda0c09b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.715+0000 7fda0be00640 1 -- 172.21.15.150:0/555571902 >> v1:172.21.15.150:6789/0 conn(0x7fda0c0703d0 legacy=0x7fda0c06d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.715+0000 7fda0be00640 1 -- 172.21.15.150:0/555571902 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c06cae0 con 0x7fda0c0ce650 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.716+0000 7fda0be00640 1 --2- 172.21.15.150:0/555571902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c09ab20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda0c15f3a0 tx=0x7fd9fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101830e408032011 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.716+0000 7fda0aa00640 1 -- 172.21.15.150:0/555571902 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc035020 con 0x7fda0c0ce650 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.716+0000 7fda0aa00640 1 -- 172.21.15.150:0/555571902 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9fc031830 con 0x7fda0c0ce650 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.716+0000 7fda0aa00640 1 -- 172.21.15.150:0/555571902 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc031c00 con 0x7fda0c0ce650 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.717+0000 7fda12e00640 1 -- 172.21.15.150:0/555571902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 msgr2=0x7fda0c09ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.717+0000 7fda12e00640 1 --2- 172.21.15.150:0/555571902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c09ab20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda0c15f3a0 tx=0x7fd9fc02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.717+0000 7fda12e00640 1 -- 172.21.15.150:0/555571902 shutdown_connections 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.717+0000 7fda12e00640 1 --2- 172.21.15.150:0/555571902 >> v2:172.21.15.187:3300/0 conn(0x7fda0c0d17c0 0x7fda0c09b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.717+0000 7fda12e00640 1 --2- 172.21.15.150:0/555571902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c09ab20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.717+0000 7fda12e00640 1 -- 172.21.15.150:0/555571902 >> 172.21.15.150:0/555571902 conn(0x7fda0c0d2720 msgr2=0x7fda0c0d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.717+0000 7fda12e00640 1 -- 172.21.15.150:0/555571902 shutdown_connections 2024-04-15T15:24:09.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.717+0000 7fda12e00640 1 -- 172.21.15.150:0/555571902 wait complete. 2024-04-15T15:24:09.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.719+0000 7fda12e00640 1 Processor -- start 2024-04-15T15:24:09.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.719+0000 7fda12e00640 1 -- start start 2024-04-15T15:24:09.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda12e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c0703d0 0x7fda0c1df060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda12e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c1e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda12e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c0d17c0 0x7fda0c0b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda12e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c09bd90 con 0x7fda0c0ce650 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda12e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c071610 con 0x7fda0c0703d0 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda12e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c0cea30 con 0x7fda0c0d17c0 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda0b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c1e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda0b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c1e84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51084/0 (socket says 172.21.15.150:51084) 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.720+0000 7fda0b400640 1 -- 172.21.15.150:0/3177385656 learned_addr learned my addr 172.21.15.150:0/3177385656 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda0be00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c0703d0 0x7fda0c1df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda10e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c0d17c0 0x7fda0c0b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda0b400640 1 -- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c0d17c0 msgr2=0x7fda0c0b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda0b400640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c0d17c0 0x7fda0c0b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda0b400640 1 -- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c0703d0 msgr2=0x7fda0c1df060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda0b400640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c0703d0 0x7fda0c1df060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda0b400640 1 -- 172.21.15.150:0/3177385656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1e7440 con 0x7fda0c0ce650 2024-04-15T15:24:09.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda10e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c0d17c0 0x7fda0c0b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:09.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda0b400640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c1e84e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800cbb0 tx=0x7fd9f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:09.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda03e00640 1 -- 172.21.15.150:0/3177385656 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8017070 con 0x7fda0c0ce650 2024-04-15T15:24:09.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda03e00640 1 -- 172.21.15.150:0/3177385656 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f800ad60 con 0x7fda0c0ce650 2024-04-15T15:24:09.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.721+0000 7fda03e00640 1 -- 172.21.15.150:0/3177385656 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8012640 con 0x7fda0c0ce650 2024-04-15T15:24:09.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.722+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c1e7720 con 0x7fda0c0ce650 2024-04-15T15:24:09.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.722+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1e7ce0 con 0x7fda0c0ce650 2024-04-15T15:24:09.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.723+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c068c80 con 0x7fda0c0ce650 2024-04-15T15:24:09.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.724+0000 7fda03e00640 1 -- 172.21.15.150:0/3177385656 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9f801a070 con 0x7fda0c0ce650 2024-04-15T15:24:09.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.725+0000 7fda03e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:09.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.725+0000 7fda03e00640 1 -- 172.21.15.150:0/3177385656 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd9f808b940 con 0x7fda0c0ce650 2024-04-15T15:24:09.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.725+0000 7fda0be00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:09.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.726+0000 7fda0be00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc036000 tx=0x7fd9fc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:09.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.729+0000 7fda03e00640 1 -- 172.21.15.150:0/3177385656 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9f8059e00 con 0x7fda0c0ce650 2024-04-15T15:24:09.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.928+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fda0c06f330 con 0x7fda0c0ce650 2024-04-15T15:24:09.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.929+0000 7fda03e00640 1 -- 172.21.15.150:0/3177385656 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fd9f8059700 con 0x7fda0c0ce650 2024-04-15T15:24:09.931 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd9ec06c8f0 msgr2=0x7fd9ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc036000 tx=0x7fd9fc008030 comp rx=0 tx=0).stop 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 msgr2=0x7fda0c1e84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c1e84e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800cbb0 tx=0x7fd9f800a430 comp rx=0 tx=0).stop 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 shutdown_connections 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c0d17c0 0x7fda0c0b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda0c0ce650 0x7fda0c1e84e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 --2- 172.21.15.150:0/3177385656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c0703d0 0x7fda0c1df060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:09.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 >> 172.21.15.150:0/3177385656 conn(0x7fda0c0d2720 msgr2=0x7fda0c1e5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:09.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 shutdown_connections 2024-04-15T15:24:09.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:09.932+0000 7fda12e00640 1 -- 172.21.15.150:0/3177385656 wait complete. 2024-04-15T15:24:09.953 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:10.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.164+0000 7f5e4b000640 1 Processor -- start 2024-04-15T15:24:10.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.165+0000 7f5e4b000640 1 -- start start 2024-04-15T15:24:10.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.166+0000 7f5e4b000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5e44162730 0x7f5e440b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.166+0000 7f5e4b000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e44162dd0 con 0x7f5e440d2b50 2024-04-15T15:24:10.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.166+0000 7f5e4b000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e441630b0 con 0x7f5e4415c680 2024-04-15T15:24:10.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.166+0000 7f5e4b000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f5e44162b90 con 0x7f5e44162730 2024-04-15T15:24:10.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.166+0000 7f5e4a600640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f5e440d2b50 0x7f5e440d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42254/0 (socket says 172.21.15.150:42254) 2024-04-15T15:24:10.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.166+0000 7f5e49200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5e44162730 0x7f5e440b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.166+0000 7f5e4a600640 1 -- 172.21.15.150:0/919268172 learned_addr learned my addr 172.21.15.150:0/919268172 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:10.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.167+0000 7f5e49200640 1 -- 172.21.15.150:0/919268172 >> v1:172.21.15.187:6789/0 conn(0x7f5e4415c680 legacy=0x7f5e4415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:10.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.167+0000 7f5e49200640 1 -- 172.21.15.150:0/919268172 >> v1:172.21.15.150:6789/0 conn(0x7f5e440d2b50 legacy=0x7f5e440d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.167+0000 7f5e43e00640 1 -- 172.21.15.150:0/919268172 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053834784 0 0) 0x7f5e44162dd0 con 0x7f5e440d2b50 2024-04-15T15:24:10.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.167+0000 7f5e49200640 1 -- 172.21.15.150:0/919268172 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e44163390 con 0x7f5e44162730 2024-04-15T15:24:10.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.168+0000 7f5e49200640 1 --2- 172.21.15.150:0/919268172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e44162730 0x7f5e440b6c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009d90 tx=0x7f5e3402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a68928da55b3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:10.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.168+0000 7f5e43e00640 1 -- 172.21.15.150:0/919268172 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34039070 con 0x7f5e44162730 2024-04-15T15:24:10.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.168+0000 7f5e43e00640 1 -- 172.21.15.150:0/919268172 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3402cc60 con 0x7f5e44162730 2024-04-15T15:24:10.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.168+0000 7f5e43e00640 1 -- 172.21.15.150:0/919268172 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34034660 con 0x7f5e44162730 2024-04-15T15:24:10.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.169+0000 7f5e4b000640 1 -- 172.21.15.150:0/919268172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e44162730 msgr2=0x7f5e440b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.169+0000 7f5e4b000640 1 --2- 172.21.15.150:0/919268172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e44162730 0x7f5e440b6c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009d90 tx=0x7f5e3402beb0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.169+0000 7f5e4b000640 1 -- 172.21.15.150:0/919268172 shutdown_connections 2024-04-15T15:24:10.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.169+0000 7f5e4b000640 1 --2- 172.21.15.150:0/919268172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e44162730 0x7f5e440b6c70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.169+0000 7f5e4b000640 1 -- 172.21.15.150:0/919268172 >> 172.21.15.150:0/919268172 conn(0x7f5e44073480 msgr2=0x7f5e44073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:10.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.169+0000 7f5e4b000640 1 -- 172.21.15.150:0/919268172 shutdown_connections 2024-04-15T15:24:10.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.169+0000 7f5e4b000640 1 -- 172.21.15.150:0/919268172 wait complete. 2024-04-15T15:24:10.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.171+0000 7f5e4b000640 1 Processor -- start 2024-04-15T15:24:10.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.171+0000 7f5e4b000640 1 -- start start 2024-04-15T15:24:10.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e4b000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440d2b50 0x7f5e440b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e4b000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4415c680 0x7f5e440b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e4b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e44162730 0x7f5e440c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e4b000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4415ff60 con 0x7f5e4415c680 2024-04-15T15:24:10.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e4b000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4415ca60 con 0x7f5e440d2b50 2024-04-15T15:24:10.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e4b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e440d1e80 con 0x7f5e44162730 2024-04-15T15:24:10.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e49200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4415c680 0x7f5e440b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e49200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4415c680 0x7f5e440b8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51108/0 (socket says 172.21.15.150:51108) 2024-04-15T15:24:10.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e49200640 1 -- 172.21.15.150:0/4291664408 learned_addr learned my addr 172.21.15.150:0/4291664408 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:10.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e49c00640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440d2b50 0x7f5e440b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e49200640 1 -- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e44162730 msgr2=0x7f5e440c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.172+0000 7f5e4a600640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e44162730 0x7f5e440c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e49200640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e44162730 0x7f5e440c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e49200640 1 -- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440d2b50 msgr2=0x7f5e440b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e49200640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440d2b50 0x7f5e440b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e49200640 1 -- 172.21.15.150:0/4291664408 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e44159a60 con 0x7f5e4415c680 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e49c00640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440d2b50 0x7f5e440b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e4a600640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e44162730 0x7f5e440c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e49200640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4415c680 0x7f5e440b8fa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009d90 tx=0x7f5e34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e41600640 1 -- 172.21.15.150:0/4291664408 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34044020 con 0x7f5e4415c680 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e41600640 1 -- 172.21.15.150:0/4291664408 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3403f670 con 0x7f5e4415c680 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e41600640 1 -- 172.21.15.150:0/4291664408 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34039040 con 0x7f5e4415c680 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e44159d40 con 0x7f5e4415c680 2024-04-15T15:24:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.173+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4415a300 con 0x7f5e4415c680 2024-04-15T15:24:10.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.175+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e440755c0 con 0x7f5e4415c680 2024-04-15T15:24:10.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.175+0000 7f5e41600640 1 -- 172.21.15.150:0/4291664408 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e34047020 con 0x7f5e4415c680 2024-04-15T15:24:10.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.177+0000 7f5e41600640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5e2806c8f0 0x7f5e2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.177+0000 7f5e41600640 1 -- 172.21.15.150:0/4291664408 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5e340ae8f0 con 0x7f5e4415c680 2024-04-15T15:24:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.177+0000 7f5e49c00640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5e2806c8f0 0x7f5e2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.178+0000 7f5e49c00640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5e2806c8f0 0x7f5e2806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5e38003eb0 tx=0x7f5e38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:10.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.182+0000 7f5e41600640 1 -- 172.21.15.150:0/4291664408 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e3404e030 con 0x7f5e4415c680 2024-04-15T15:24:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.386+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5e4406dbe0 con 0x7f5e4415c680 2024-04-15T15:24:10.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.387+0000 7f5e41600640 1 -- 172.21.15.150:0/4291664408 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f5e3407cbf0 con 0x7f5e4415c680 2024-04-15T15:24:10.389 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5e2806c8f0 msgr2=0x7f5e2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5e2806c8f0 0x7f5e2806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5e38003eb0 tx=0x7f5e38009040 comp rx=0 tx=0).stop 2024-04-15T15:24:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4415c680 msgr2=0x7f5e440b8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4415c680 0x7f5e440b8fa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009d90 tx=0x7f5e34004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 shutdown_connections 2024-04-15T15:24:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5e2806c8f0 0x7f5e2806edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e44162730 0x7f5e440c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4415c680 0x7f5e440b8fa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 --2- 172.21.15.150:0/4291664408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440d2b50 0x7f5e440b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.390+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 >> 172.21.15.150:0/4291664408 conn(0x7f5e44073480 msgr2=0x7f5e4415f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.391+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 shutdown_connections 2024-04-15T15:24:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.391+0000 7f5e4b000640 1 -- 172.21.15.150:0/4291664408 wait complete. 2024-04-15T15:24:10.410 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:24:10.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.614+0000 7f9d93200640 1 Processor -- start 2024-04-15T15:24:10.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.614+0000 7f9d93200640 1 -- start start 2024-04-15T15:24:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.615+0000 7f9d93200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d9415b860 0x7f9d9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.615+0000 7f9d93200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d940cdfb0 con 0x7f9d9406f320 2024-04-15T15:24:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.615+0000 7f9d93200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d9415f2b0 con 0x7f9d9415b480 2024-04-15T15:24:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.615+0000 7f9d93200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9d9415f060 con 0x7f9d9415b860 2024-04-15T15:24:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.616+0000 7f9d92800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9d9406f320 0x7f9d940b3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42270/0 (socket says 172.21.15.150:42270) 2024-04-15T15:24:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.616+0000 7f9d92800640 1 -- 172.21.15.150:0/790092645 learned_addr learned my addr 172.21.15.150:0/790092645 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:10.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.616+0000 7f9d89400640 1 --2- 172.21.15.150:0/790092645 >> v2:172.21.15.187:3300/0 conn(0x7f9d9415b860 0x7f9d9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.616+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034791600 0 0) 0x7f9d940cdfb0 con 0x7f9d9406f320 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.616+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d78003520 con 0x7f9d9406f320 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3038693868 0 0) 0x7f9d78003520 con 0x7f9d9406f320 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 >> v2:172.21.15.187:3300/0 conn(0x7f9d9415b860 msgr2=0x7f9d9406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 --2- 172.21.15.150:0/790092645 >> v2:172.21.15.187:3300/0 conn(0x7f9d9415b860 0x7f9d9406ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 >> v1:172.21.15.187:6789/0 conn(0x7f9d9415b480 legacy=0x7f9d940cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9415f590 con 0x7f9d9406f320 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f9d8c0019f0 con 0x7f9d9406f320 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 >> v1:172.21.15.150:6789/0 conn(0x7f9d9406f320 legacy=0x7f9d940b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d78004800 0x7f9d78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d780051c0 0x7f9d7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9415f060 con 0x7f9d780039b0 2024-04-15T15:24:10.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7800c060 con 0x7f9d780051c0 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7800c260 con 0x7f9d78004800 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91e00640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d92800640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d780051c0 0x7f9d7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d89400640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d78004800 0x7f9d78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91e00640 1 -- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d78004800 msgr2=0x7f9d78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91e00640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d78004800 0x7f9d78004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91e00640 1 -- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d780051c0 msgr2=0x7f9d7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91e00640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d780051c0 0x7f9d7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.617+0000 7f9d91e00640 1 -- 172.21.15.150:0/790092645 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d78003520 con 0x7f9d780039b0 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.618+0000 7f9d92800640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d780051c0 0x7f9d7800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:10.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.618+0000 7f9d92800640 1 -- 172.21.15.150:0/790092645 reap_dead start 2024-04-15T15:24:10.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.619+0000 7f9d91e00640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d78003dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9d84001750 tx=0x7f9d78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6221e39bb5460a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.619+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84001370 con 0x7f9d780039b0 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.619+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d840052d0 con 0x7f9d780039b0 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.619+0000 7f9d91400640 1 -- 172.21.15.150:0/790092645 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84003830 con 0x7f9d780039b0 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.619+0000 7f9d93200640 1 -- 172.21.15.150:0/790092645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 msgr2=0x7f9d78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.619+0000 7f9d93200640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d78003dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9d84001750 tx=0x7f9d78005830 comp rx=0 tx=0).stop 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.620+0000 7f9d93200640 1 -- 172.21.15.150:0/790092645 shutdown_connections 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.620+0000 7f9d93200640 1 --2- 172.21.15.150:0/790092645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d78003dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.620+0000 7f9d93200640 1 -- 172.21.15.150:0/790092645 >> 172.21.15.150:0/790092645 conn(0x7f9d9406b930 msgr2=0x7f9d9406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.620+0000 7f9d93200640 1 -- 172.21.15.150:0/790092645 shutdown_connections 2024-04-15T15:24:10.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.620+0000 7f9d93200640 1 -- 172.21.15.150:0/790092645 wait complete. 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.621+0000 7f9d93200640 1 Processor -- start 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.622+0000 7f9d93200640 1 -- start start 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.622+0000 7f9d93200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d94151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.622+0000 7f9d93200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d94150bf0 0x7f9d9414ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.622+0000 7f9d93200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d941533d0 0x7f9d94153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d93200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9415fb50 con 0x7f9d780039b0 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d93200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9415fec0 con 0x7f9d94150bf0 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d93200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d940cd550 con 0x7f9d941533d0 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d94151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d94151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51136/0 (socket says 172.21.15.150:51136) 2024-04-15T15:24:10.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 -- 172.21.15.150:0/3937509384 learned_addr learned my addr 172.21.15.150:0/3937509384 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:10.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 -- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d941533d0 msgr2=0x7f9d94153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d941533d0 0x7f9d94153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 -- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d94150bf0 msgr2=0x7f9d9414ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d94150bf0 0x7f9d9414ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 -- 172.21.15.150:0/3937509384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d941f3d20 con 0x7f9d780039b0 2024-04-15T15:24:10.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.623+0000 7f9d91e00640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d94151880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d84005040 tx=0x7f9d84001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:10.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.624+0000 7f9d8aa00640 1 -- 172.21.15.150:0/3937509384 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84041070 con 0x7f9d780039b0 2024-04-15T15:24:10.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.624+0000 7f9d8aa00640 1 -- 172.21.15.150:0/3937509384 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d84005480 con 0x7f9d780039b0 2024-04-15T15:24:10.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.624+0000 7f9d8aa00640 1 -- 172.21.15.150:0/3937509384 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84007a30 con 0x7f9d780039b0 2024-04-15T15:24:10.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.624+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d94151070 con 0x7f9d780039b0 2024-04-15T15:24:10.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.624+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941f3f90 con 0x7f9d780039b0 2024-04-15T15:24:10.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.625+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d940755c0 con 0x7f9d780039b0 2024-04-15T15:24:10.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.626+0000 7f9d8aa00640 1 -- 172.21.15.150:0/3937509384 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d84044070 con 0x7f9d780039b0 2024-04-15T15:24:10.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.627+0000 7f9d8aa00640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:10.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.627+0000 7f9d8aa00640 1 -- 172.21.15.150:0/3937509384 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9d840ad680 con 0x7f9d780039b0 2024-04-15T15:24:10.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.627+0000 7f9d89400640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:10.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.628+0000 7f9d89400640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7006c8f0 0x7f9d7006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d9414ef90 tx=0x7f9d7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.631+0000 7f9d8aa00640 1 -- 172.21.15.150:0/3937509384 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d8407baa0 con 0x7f9d780039b0 2024-04-15T15:24:10.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.871+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d940b3930 con 0x7f9d780039b0 2024-04-15T15:24:10.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.872+0000 7f9d8aa00640 1 -- 172.21.15.150:0/3937509384 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f9d8407b3a0 con 0x7f9d780039b0 2024-04-15T15:24:10.874 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:10.874 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:23:54.683053+0000","modified":"2024-04-15T15:23:57.779875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4250,"qdb_cluster":[4250]},"id":1}]} 2024-04-15T15:24:10.874 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-04-15T15:24:10.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.874+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7006c8f0 msgr2=0x7f9d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.874+0000 7f9d93200640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7006c8f0 0x7f9d7006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d9414ef90 tx=0x7f9d7c008040 comp rx=0 tx=0).stop 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.874+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 msgr2=0x7f9d94151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.874+0000 7f9d93200640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d94151880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d84005040 tx=0x7f9d84001120 comp rx=0 tx=0).stop 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.875+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 shutdown_connections 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.875+0000 7f9d93200640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.875+0000 7f9d93200640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d941533d0 0x7f9d94153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.875+0000 7f9d93200640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d94150bf0 0x7f9d9414ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.875+0000 7f9d93200640 1 --2- 172.21.15.150:0/3937509384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d780039b0 0x7f9d94151880 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.875+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 >> 172.21.15.150:0/3937509384 conn(0x7f9d9406b930 msgr2=0x7f9d9415e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.875+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 shutdown_connections 2024-04-15T15:24:10.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:10.875+0000 7f9d93200640 1 -- 172.21.15.150:0/3937509384 wait complete. 2024-04-15T15:24:10.895 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:11.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.099+0000 7fd907e00640 1 Processor -- start 2024-04-15T15:24:11.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.100+0000 7fd907e00640 1 -- start start 2024-04-15T15:24:11.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.101+0000 7fd907e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd90015c890 0x7fd90009b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.101+0000 7fd907e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd9000b6210 0x7fd9000b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.101+0000 7fd907e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9000b27c0 con 0x7fd9000b2270 2024-04-15T15:24:11.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.101+0000 7fd907e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fd9000b2aa0 con 0x7fd90015c890 2024-04-15T15:24:11.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.101+0000 7fd907e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd9000b2d80 con 0x7fd9000b6210 2024-04-15T15:24:11.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.102+0000 7fd905400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd90015c890 0x7fd90009b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.102+0000 7fd905400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd90015c890 0x7fd90009b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51150/0 (socket says 172.21.15.150:51150) 2024-04-15T15:24:11.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.102+0000 7fd905400640 1 -- 172.21.15.150:0/2229497797 learned_addr learned my addr 172.21.15.150:0/2229497797 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:11.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.102+0000 7fd905400640 1 -- 172.21.15.150:0/2229497797 >> v2:172.21.15.187:3300/0 conn(0x7fd9000b6210 msgr2=0x7fd9000b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:11.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.102+0000 7fd905400640 1 --2- 172.21.15.150:0/2229497797 >> v2:172.21.15.187:3300/0 conn(0x7fd9000b6210 0x7fd9000b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.102+0000 7fd905400640 1 -- 172.21.15.150:0/2229497797 >> v1:172.21.15.187:6789/0 conn(0x7fd9000b2270 legacy=0x7fd9000afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:11.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.102+0000 7fd905400640 1 -- 172.21.15.150:0/2229497797 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9000b3060 con 0x7fd90015c890 2024-04-15T15:24:11.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.103+0000 7fd905400640 1 --2- 172.21.15.150:0/2229497797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90015c890 0x7fd90009b610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9000733e0 tx=0x7fd8f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ac4f4c5a668eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:11.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.103+0000 7fd8ffe00640 1 -- 172.21.15.150:0/2229497797 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0035020 con 0x7fd90015c890 2024-04-15T15:24:11.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.103+0000 7fd8ffe00640 1 -- 172.21.15.150:0/2229497797 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f0031870 con 0x7fd90015c890 2024-04-15T15:24:11.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.103+0000 7fd8ffe00640 1 -- 172.21.15.150:0/2229497797 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0031c40 con 0x7fd90015c890 2024-04-15T15:24:11.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.104+0000 7fd907e00640 1 -- 172.21.15.150:0/2229497797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90015c890 msgr2=0x7fd90009b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.104+0000 7fd907e00640 1 --2- 172.21.15.150:0/2229497797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90015c890 0x7fd90009b610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9000733e0 tx=0x7fd8f002b910 comp rx=0 tx=0).stop 2024-04-15T15:24:11.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.104+0000 7fd907e00640 1 -- 172.21.15.150:0/2229497797 shutdown_connections 2024-04-15T15:24:11.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.104+0000 7fd907e00640 1 --2- 172.21.15.150:0/2229497797 >> v2:172.21.15.187:3300/0 conn(0x7fd9000b6210 0x7fd9000b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.104+0000 7fd907e00640 1 --2- 172.21.15.150:0/2229497797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd90015c890 0x7fd90009b610 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.104+0000 7fd907e00640 1 -- 172.21.15.150:0/2229497797 >> 172.21.15.150:0/2229497797 conn(0x7fd9000b4400 msgr2=0x7fd9000b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:11.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.104+0000 7fd907e00640 1 -- 172.21.15.150:0/2229497797 shutdown_connections 2024-04-15T15:24:11.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.104+0000 7fd907e00640 1 -- 172.21.15.150:0/2229497797 wait complete. 2024-04-15T15:24:11.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.106+0000 7fd907e00640 1 Processor -- start 2024-04-15T15:24:11.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.107+0000 7fd907e00640 1 -- start start 2024-04-15T15:24:11.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd907e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9000b2270 0x7fd9001562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd907e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000b6210 0x7fd90009f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd907e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd90015c890 0x7fd90009f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd907e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd90015cc70 con 0x7fd9000b2270 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd907e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9000b5c40 con 0x7fd9000b6210 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd907e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd9000b0550 con 0x7fd90015c890 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd905400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9000b2270 0x7fd9001562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd905400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9000b2270 0x7fd9001562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51160/0 (socket says 172.21.15.150:51160) 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.108+0000 7fd905400640 1 -- 172.21.15.150:0/4179141764 learned_addr learned my addr 172.21.15.150:0/4179141764 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd905400640 1 -- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd90015c890 msgr2=0x7fd90009f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd905e00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd90015c890 0x7fd90009f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd904a00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000b6210 0x7fd90009f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd905400640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd90015c890 0x7fd90009f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd905400640 1 -- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000b6210 msgr2=0x7fd90009f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd905400640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000b6210 0x7fd90009f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd905400640 1 -- 172.21.15.150:0/4179141764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90015e3f0 con 0x7fd9000b2270 2024-04-15T15:24:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd905e00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd90015c890 0x7fd90009f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd904a00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000b6210 0x7fd90009f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd905400640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9000b2270 0x7fd9001562d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0000c00 tx=0x7fd8f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd8fd600640 1 -- 172.21.15.150:0/4179141764 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f003ad60 con 0x7fd9000b2270 2024-04-15T15:24:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd8fd600640 1 -- 172.21.15.150:0/4179141764 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f00443f0 con 0x7fd9000b2270 2024-04-15T15:24:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd8fd600640 1 -- 172.21.15.150:0/4179141764 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00447e0 con 0x7fd9000b2270 2024-04-15T15:24:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.109+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90015e690 con 0x7fd9000b2270 2024-04-15T15:24:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.110+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90015ec50 con 0x7fd9000b2270 2024-04-15T15:24:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.111+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd900068c80 con 0x7fd9000b2270 2024-04-15T15:24:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.111+0000 7fd8fd600640 1 -- 172.21.15.150:0/4179141764 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8f0038020 con 0x7fd9000b2270 2024-04-15T15:24:11.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.113+0000 7fd8fd600640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8e406c8f0 0x7fd8e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.113+0000 7fd8fd600640 1 -- 172.21.15.150:0/4179141764 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd8f00ade20 con 0x7fd9000b2270 2024-04-15T15:24:11.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.113+0000 7fd904a00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8e406c8f0 0x7fd8e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.114+0000 7fd904a00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8e406c8f0 0x7fd8e406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd900155a00 tx=0x7fd8f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:11.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.117+0000 7fd8fd600640 1 -- 172.21.15.150:0/4179141764 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8f003f070 con 0x7fd9000b2270 2024-04-15T15:24:11.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.325+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9000b52d0 con 0x7fd9000b2270 2024-04-15T15:24:11.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.327+0000 7fd8fd600640 1 -- 172.21.15.150:0/4179141764 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7fd8f007c1f0 con 0x7fd9000b2270 2024-04-15T15:24:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:11.329 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:11.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8e406c8f0 msgr2=0x7fd8e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8e406c8f0 0x7fd8e406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd900155a00 tx=0x7fd8f4009040 comp rx=0 tx=0).stop 2024-04-15T15:24:11.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9000b2270 msgr2=0x7fd9001562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9000b2270 0x7fd9001562d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0000c00 tx=0x7fd8f00046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 shutdown_connections 2024-04-15T15:24:11.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd8e406c8f0 0x7fd8e406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd90015c890 0x7fd90009f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9000b6210 0x7fd90009f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 --2- 172.21.15.150:0/4179141764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9000b2270 0x7fd9001562d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.329+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 >> 172.21.15.150:0/4179141764 conn(0x7fd9000b4400 msgr2=0x7fd9000d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:11.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.330+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 shutdown_connections 2024-04-15T15:24:11.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.330+0000 7fd907e00640 1 -- 172.21.15.150:0/4179141764 wait complete. 2024-04-15T15:24:11.350 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:24:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.552+0000 7f2608a00640 1 Processor -- start 2024-04-15T15:24:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.552+0000 7f2608a00640 1 -- start start 2024-04-15T15:24:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.554+0000 7f2608a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2604162020 0x7f2604162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.554+0000 7f2608a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040d1c90 0x7f26040d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.554+0000 7f2608a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26040d0090 con 0x7f26040cf110 2024-04-15T15:24:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.554+0000 7f2608a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2604163440 con 0x7f2604162020 2024-04-15T15:24:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.554+0000 7f2608a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f26040b7b30 con 0x7f26040d1c90 2024-04-15T15:24:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.554+0000 7f2603e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040d1c90 0x7f26040d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.554+0000 7f2603400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2604162020 0x7f2604162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.554+0000 7f2602a00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f26040cf110 0x7f26040d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:33506/0 (socket says 172.21.15.150:33506) 2024-04-15T15:24:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.555+0000 7f2602a00640 1 -- 172.21.15.150:0/4172581428 learned_addr learned my addr 172.21.15.150:0/4172581428 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.555+0000 7f2603e00640 1 -- 172.21.15.150:0/4172581428 >> v2:172.21.15.187:3300/0 conn(0x7f2604162020 msgr2=0x7f2604162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.555+0000 7f2603e00640 1 --2- 172.21.15.150:0/4172581428 >> v2:172.21.15.187:3300/0 conn(0x7f2604162020 0x7f2604162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.555+0000 7f2603e00640 1 -- 172.21.15.150:0/4172581428 >> v1:172.21.15.187:6789/0 conn(0x7f26040cf110 legacy=0x7f26040d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.555+0000 7f2603e00640 1 -- 172.21.15.150:0/4172581428 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26040cfad0 con 0x7f26040d1c90 2024-04-15T15:24:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.556+0000 7f2603e00640 1 --2- 172.21.15.150:0/4172581428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040d1c90 0x7f26040d3390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0097a0 tx=0x7f25fc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ba440dc2acea24 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2602000640 1 -- 172.21.15.150:0/4172581428 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc035020 con 0x7f26040d1c90 2024-04-15T15:24:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2602000640 1 -- 172.21.15.150:0/4172581428 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25fc031830 con 0x7f26040d1c90 2024-04-15T15:24:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2602000640 1 -- 172.21.15.150:0/4172581428 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc031c00 con 0x7f26040d1c90 2024-04-15T15:24:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2608a00640 1 -- 172.21.15.150:0/4172581428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040d1c90 msgr2=0x7f26040d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2608a00640 1 --2- 172.21.15.150:0/4172581428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040d1c90 0x7f26040d3390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0097a0 tx=0x7f25fc02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2608a00640 1 -- 172.21.15.150:0/4172581428 shutdown_connections 2024-04-15T15:24:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2608a00640 1 --2- 172.21.15.150:0/4172581428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040d1c90 0x7f26040d3390 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2608a00640 1 --2- 172.21.15.150:0/4172581428 >> v2:172.21.15.187:3300/0 conn(0x7f2604162020 0x7f2604162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.557+0000 7f2608a00640 1 -- 172.21.15.150:0/4172581428 >> 172.21.15.150:0/4172581428 conn(0x7f260406cf00 msgr2=0x7f260406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.558+0000 7f2608a00640 1 -- 172.21.15.150:0/4172581428 shutdown_connections 2024-04-15T15:24:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.558+0000 7f2608a00640 1 -- 172.21.15.150:0/4172581428 wait complete. 2024-04-15T15:24:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.559+0000 7f2608a00640 1 Processor -- start 2024-04-15T15:24:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.559+0000 7f2608a00640 1 -- start start 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.560+0000 7f2608a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040cf110 0x7f26041f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.560+0000 7f2608a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26040d1c90 0x7f26040ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.560+0000 7f2608a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604162020 0x7f26041571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.560+0000 7f2608a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26040d2b90 con 0x7f26040d1c90 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.560+0000 7f2608a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f26040cf880 con 0x7f2604162020 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.560+0000 7f2608a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f26040d2e70 con 0x7f26040cf110 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26040d1c90 0x7f26040ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26040d1c90 0x7f26040ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51168/0 (socket says 172.21.15.150:51168) 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 -- 172.21.15.150:0/2098721373 learned_addr learned my addr 172.21.15.150:0/2098721373 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2603400640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040cf110 0x7f26041f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2603e00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604162020 0x7f26041571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 -- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040cf110 msgr2=0x7f26041f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040cf110 0x7f26041f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 -- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604162020 msgr2=0x7f26041571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604162020 0x7f26041571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 -- 172.21.15.150:0/2098721373 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26040cc870 con 0x7f26040d1c90 2024-04-15T15:24:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.561+0000 7f2602a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26040d1c90 0x7f26040ce120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c440 tx=0x7f25f000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:11.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.562+0000 7f25fb400640 1 -- 172.21.15.150:0/2098721373 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0017070 con 0x7f26040d1c90 2024-04-15T15:24:11.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.562+0000 7f25fb400640 1 -- 172.21.15.150:0/2098721373 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f0005ce0 con 0x7f26040d1c90 2024-04-15T15:24:11.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.562+0000 7f25fb400640 1 -- 172.21.15.150:0/2098721373 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00072f0 con 0x7f26040d1c90 2024-04-15T15:24:11.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.562+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26040ccb50 con 0x7f26040d1c90 2024-04-15T15:24:11.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.562+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26040cd110 con 0x7f26040d1c90 2024-04-15T15:24:11.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.563+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26041f4ed0 con 0x7f26040d1c90 2024-04-15T15:24:11.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.564+0000 7f25fb400640 1 -- 172.21.15.150:0/2098721373 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25f001a070 con 0x7f26040d1c90 2024-04-15T15:24:11.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.565+0000 7f25fb400640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f25e406c8f0 0x7f25e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:11.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.565+0000 7f25fb400640 1 -- 172.21.15.150:0/2098721373 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f25f008af20 con 0x7f26040d1c90 2024-04-15T15:24:11.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.565+0000 7f2603400640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f25e406c8f0 0x7f25e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:11.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.566+0000 7f2603400640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f25e406c8f0 0x7f25e406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f25f40074a0 tx=0x7f25f400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:11.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.569+0000 7f25fb400640 1 -- 172.21.15.150:0/2098721373 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26041f4ed0 con 0x7f26040d1c90 2024-04-15T15:24:11.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.812+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26040755c0 con 0x7f26040d1c90 2024-04-15T15:24:11.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.813+0000 7f25fb400640 1 -- 172.21.15.150:0/2098721373 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f26040d0090 con 0x7f26040d1c90 2024-04-15T15:24:11.814 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:11.815 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:23:54.683053+0000","modified":"2024-04-15T15:23:57.779875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4250,"qdb_cluster":[4250]},"id":1}]} 2024-04-15T15:24:11.815 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-04-15T15:24:11.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.815+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f25e406c8f0 msgr2=0x7f25e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f25e406c8f0 0x7f25e406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f25f40074a0 tx=0x7f25f400b040 comp rx=0 tx=0).stop 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26040d1c90 msgr2=0x7f26040ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26040d1c90 0x7f26040ce120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c440 tx=0x7f25f000c900 comp rx=0 tx=0).stop 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 shutdown_connections 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604162020 0x7f26041571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26040d1c90 0x7f26040ce120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f25e406c8f0 0x7f25e406edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 --2- 172.21.15.150:0/2098721373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26040cf110 0x7f26041f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 >> 172.21.15.150:0/2098721373 conn(0x7f260406cf00 msgr2=0x7f26040b7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 shutdown_connections 2024-04-15T15:24:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:11.816+0000 7f2608a00640 1 -- 172.21.15.150:0/2098721373 wait complete. 2024-04-15T15:24:12.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:23:54.683053+0000', 'modified': '2024-04-15T15:23:57.779875+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4250}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4250': {'gid': 4250, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6837/3441543802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6835', 'nonce': 3441543802}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 3441543802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4250, 'qdb_cluster': [4250]} 2024-04-15T15:24:12.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:24:12.240 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-15T15:24:12.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.458+0000 7fcb3dc00640 1 Processor -- start 2024-04-15T15:24:12.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.458+0000 7fcb3dc00640 1 -- start start 2024-04-15T15:24:12.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.460+0000 7fcb3dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb3813f6a0 0x7fcb380ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.460+0000 7fcb3dc00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3806ef20 con 0x7fcb38072580 2024-04-15T15:24:12.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.460+0000 7fcb3dc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3815d200 con 0x7fcb38072100 2024-04-15T15:24:12.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.460+0000 7fcb3dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb380a95b0 con 0x7fcb3813f6a0 2024-04-15T15:24:12.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.460+0000 7fcb37400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fcb38072580 0x7fcb3806e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42282/0 (socket says 172.21.15.150:42282) 2024-04-15T15:24:12.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.460+0000 7fcb37400640 1 -- 172.21.15.150:0/3365302100 learned_addr learned my addr 172.21.15.150:0/3365302100 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:12.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb36a00640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb3813f6a0 0x7fcb380ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 23154368 0 0) 0x7fcb3806ef20 con 0x7fcb38072580 2024-04-15T15:24:12.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb1c003520 con 0x7fcb38072580 2024-04-15T15:24:12.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1953617619 0 0) 0x7fcb1c003520 con 0x7fcb38072580 2024-04-15T15:24:12.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb3813f6a0 msgr2=0x7fcb380ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb3813f6a0 0x7fcb380ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 >> v1:172.21.15.187:6789/0 conn(0x7fcb38072100 legacy=0x7fcb3806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3815d4e0 con 0x7fcb38072580 2024-04-15T15:24:12.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fcb300019f0 con 0x7fcb38072580 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 >> v1:172.21.15.150:6789/0 conn(0x7fcb38072580 legacy=0x7fcb3806e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb1c0039e0 0x7fcb1c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb3813f6a0 0x7fcb1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c00ba70 con 0x7fcb1c004800 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c00bbf0 con 0x7fcb1c0039e0 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.461+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1c00bdf0 con 0x7fcb3813f6a0 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.462+0000 7fcb36000640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.462+0000 7fcb36a00640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb1c0039e0 0x7fcb1c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.462+0000 7fcb37400640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb3813f6a0 0x7fcb1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.462+0000 7fcb36000640 1 -- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb3813f6a0 msgr2=0x7fcb1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.462+0000 7fcb36000640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb3813f6a0 0x7fcb1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.462+0000 7fcb36000640 1 -- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb1c0039e0 msgr2=0x7fcb1c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.462+0000 7fcb36000640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb1c0039e0 0x7fcb1c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.462+0000 7fcb36000640 1 -- 172.21.15.150:0/3365302100 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c003520 con 0x7fcb1c004800 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.463+0000 7fcb36000640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb1c004c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcb20001720 tx=0x7fcb1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5a24c628e1680 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.463+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20001370 con 0x7fcb1c004800 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.463+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb200052d0 con 0x7fcb1c004800 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.463+0000 7fcb35600640 1 -- 172.21.15.150:0/3365302100 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20003830 con 0x7fcb1c004800 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.464+0000 7fcb3dc00640 1 -- 172.21.15.150:0/3365302100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 msgr2=0x7fcb1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.464+0000 7fcb3dc00640 1 --2- 172.21.15.150:0/3365302100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb1c004c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcb20001720 tx=0x7fcb1c005830 comp rx=0 tx=0).stop 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.464+0000 7fcb37400640 1 -- 172.21.15.150:0/3365302100 reap_dead start 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.464+0000 7fcb3dc00640 1 -- 172.21.15.150:0/3365302100 shutdown_connections 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.464+0000 7fcb3dc00640 1 -- 172.21.15.150:0/3365302100 >> 172.21.15.150:0/3365302100 conn(0x7fcb380b2460 msgr2=0x7fcb380b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.464+0000 7fcb3dc00640 1 -- 172.21.15.150:0/3365302100 shutdown_connections 2024-04-15T15:24:12.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.464+0000 7fcb3dc00640 1 -- 172.21.15.150:0/3365302100 wait complete. 2024-04-15T15:24:12.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.466+0000 7fcb3dc00640 1 Processor -- start 2024-04-15T15:24:12.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.466+0000 7fcb3dc00640 1 -- start start 2024-04-15T15:24:12.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb3dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb3813b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb3dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb381f5bc0 0x7fcb381f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb3dc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb38159030 0x7fcb381594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb3dc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb380729f0 con 0x7fcb1c004800 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb3dc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38070400 con 0x7fcb38159030 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb3dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3813fef0 con 0x7fcb381f5bc0 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb36a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb3813b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb36a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb3813b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51186/0 (socket says 172.21.15.150:51186) 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.467+0000 7fcb36a00640 1 -- 172.21.15.150:0/1141818432 learned_addr learned my addr 172.21.15.150:0/1141818432 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb36000640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb381f5bc0 0x7fcb381f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb37400640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb38159030 0x7fcb381594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb36a00640 1 -- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb381f5bc0 msgr2=0x7fcb381f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb36a00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb381f5bc0 0x7fcb381f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb36a00640 1 -- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb38159030 msgr2=0x7fcb381594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb36a00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb38159030 0x7fcb381594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb36a00640 1 -- 172.21.15.150:0/1141818432 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb381f36e0 con 0x7fcb1c004800 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb37400640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb38159030 0x7fcb381594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb36a00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb3813b3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb28011a00 tx=0x7fcb28011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb2ea00640 1 -- 172.21.15.150:0/1141818432 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800faf0 con 0x7fcb1c004800 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb2ea00640 1 -- 172.21.15.150:0/1141818432 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb2800fc90 con 0x7fcb1c004800 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb2ea00640 1 -- 172.21.15.150:0/1141818432 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28014440 con 0x7fcb1c004800 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb3dc00640 1 -- 172.21.15.150:0/1141818432 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3813b930 con 0x7fcb1c004800 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.468+0000 7fcb3dc00640 1 -- 172.21.15.150:0/1141818432 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb381f3c30 con 0x7fcb1c004800 2024-04-15T15:24:12.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.471+0000 7fcb2ea00640 1 -- 172.21.15.150:0/1141818432 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb28012040 con 0x7fcb1c004800 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.472+0000 7fcb2ea00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb1406c920 0x7fcb1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.472+0000 7fcb2ea00640 1 -- 172.21.15.150:0/1141818432 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcb2808c280 con 0x7fcb1c004800 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.472+0000 7fcb40982740 1 Processor -- start 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.472+0000 7fcb36000640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb1406c920 0x7fcb1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.472+0000 7fcb40982740 1 -- start start 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.473+0000 7fcb40982740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770386e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.473+0000 7fcb40982740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556876ff2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.473+0000 7fcb40982740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556876ff27c0 0x556876ff0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.473+0000 7fcb40982740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556876f579c0 con 0x5568770195a0 2024-04-15T15:24:12.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.473+0000 7fcb40982740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556876efe560 con 0x556876ff27c0 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.473+0000 7fcb40982740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556876f53980 con 0x556877019980 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.473+0000 7fcb36000640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb1406c920 0x7fcb1406ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb20033000 tx=0x7fcb20009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770386e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51196/0 (socket says 172.21.15.150:51196) 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 -- 172.21.15.150:0/467238127 learned_addr learned my addr 172.21.15.150:0/467238127 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36000640 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556876ff2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb37400640 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556876ff27c0 0x556876ff0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 -- 172.21.15.150:0/467238127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 msgr2=0x556876ff2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556876ff2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 -- 172.21.15.150:0/467238127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556876ff27c0 msgr2=0x556876ff0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556876ff27c0 0x556876ff0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 -- 172.21.15.150:0/467238127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556876c6a2d0 con 0x5568770195a0 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb36a00640 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770386e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcb280119d0 tx=0x7fcb28062c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49da8965ad7bfa31 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb27400640 1 -- 172.21.15.150:0/467238127 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28022030 con 0x5568770195a0 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb27400640 1 -- 172.21.15.150:0/467238127 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb280580e0 con 0x5568770195a0 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.474+0000 7fcb27400640 1 -- 172.21.15.150:0/467238127 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2805a440 con 0x5568770195a0 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 -- 172.21.15.150:0/467238127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 msgr2=0x5568770386e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770386e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcb280119d0 tx=0x7fcb28062c30 comp rx=0 tx=0).stop 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 -- 172.21.15.150:0/467238127 shutdown_connections 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556876ff27c0 0x556876ff0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556876ff2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 --2- 172.21.15.150:0/467238127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770386e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 -- 172.21.15.150:0/467238127 >> 172.21.15.150:0/467238127 conn(0x556876fee860 msgr2=0x5568770373e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 -- 172.21.15.150:0/467238127 shutdown_connections 2024-04-15T15:24:12.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.475+0000 7fcb40982740 1 -- 172.21.15.150:0/467238127 wait complete. 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.476+0000 7fcb40982740 1 -- start start 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.476+0000 7fcb40982740 1 Processor -- start 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb40982740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 0x5568770348e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb40982740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556877031c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb40982740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb40982740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556876fec690 con 0x556877036800 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb40982740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556876f11920 con 0x5568770195a0 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb40982740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556876ff2c40 con 0x556877019980 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb40982740 10 client.0 _ll_register_callbacks cb 0x556876f11310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb37400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb37400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770321c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51210/0 (socket says 172.21.15.150:51210) 2024-04-15T15:24:12.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.477+0000 7fcb37400640 1 -- 172.21.15.150:0/2013449819 learned_addr learned my addr 172.21.15.150:0/2013449819 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb36000640 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556877031c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb36a00640 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 0x5568770348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb37400640 1 -- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 msgr2=0x556877031c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb37400640 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556877031c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb37400640 1 -- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 msgr2=0x5568770348e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb37400640 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 0x5568770348e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb37400640 1 -- 172.21.15.150:0/2013449819 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556876d346b0 con 0x556877036800 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb37400640 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770321c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb380cf7f0 tx=0x7fcb30007a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ee6d5e42e6762f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb1be00640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb1be00640 1 -- 172.21.15.150:0/2013449819 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000d030 con 0x556877036800 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb1be00640 1 -- 172.21.15.150:0/2013449819 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcb30004d60 con 0x556877036800 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.478+0000 7fcb40982740 1 -- 172.21.15.150:0/2013449819 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556876efdb70 con 0x556877036800 2024-04-15T15:24:12.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb1be00640 1 -- 172.21.15.150:0/2013449819 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fcb30010040 con 0x556877036800 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb40982740 10 client.4659 fetch_fsmap learned FSMap version 8 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb40982740 1 -- 172.21.15.150:0/2013449819 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55687701add0 con 0x556877036800 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb1be00640 1 -- 172.21.15.150:0/2013449819 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556876efdb70 con 0x556877036800 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb1be00640 1 -- 172.21.15.150:0/2013449819 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fcb3001d050 con 0x556877036800 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb40982740 10 client.4659 fetch_fsmap finished waiting for FSMap version 8 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb40982740 10 client.4659 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb40982740 20 client.4659 populate_metadata read hostname 'smithi150' 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb40982740 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] conn(0x556877048080 0x556877056410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb40982740 4 client.4659 mds_command: new command op to 4250 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:24:12.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.479+0000 7fcb40982740 1 -- 172.21.15.150:0/2013449819 --> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556876f653e0 con 0x556877048080 2024-04-15T15:24:12.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.480+0000 7fcb36a00640 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] conn(0x556877048080 0x556877056410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.481+0000 7fcb36a00640 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] conn(0x556877048080 0x556877056410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef5a63ed714732ce server_cookie=c5f68b6de2a0b1b3 in_seq=0 out_seq=0 2024-04-15T15:24:12.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.481+0000 7fcb1be00640 10 client.4659 ms_handle_connect on v2:172.21.15.187:6835/3441543802 2024-04-15T15:24:12.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.482+0000 7fcb1be00640 1 -- 172.21.15.150:0/2013449819 <== mds.0 v2:172.21.15.187:6835/3441543802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556876f653e0 con 0x556877048080 2024-04-15T15:24:12.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.482+0000 7fcb1be00640 10 client.4659 handle_command_reply: tid=0 2024-04-15T15:24:12.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.482+0000 7fcb40982740 1 client.4659 shutdown 2024-04-15T15:24:12.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.482+0000 7fcb40982740 1 -- 172.21.15.150:0/2013449819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 msgr2=0x5568770321c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.482+0000 7fcb40982740 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770321c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb380cf7f0 tx=0x7fcb30007a70 comp rx=0 tx=0).stop 2024-04-15T15:24:12.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.482+0000 7fcb40982740 1 -- 172.21.15.150:0/2013449819 shutdown_connections 2024-04-15T15:24:12.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.482+0000 7fcb40982740 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] conn(0x556877048080 0x556877056410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.482+0000 7fcb40982740 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770321c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.483+0000 7fcb40982740 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556877031c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.483+0000 7fcb40982740 1 --2- 172.21.15.150:0/2013449819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 0x5568770348e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.483+0000 7fcb40982740 1 -- 172.21.15.150:0/2013449819 >> 172.21.15.150:0/2013449819 conn(0x556876ff27c0 msgr2=0x556877037370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.483+0000 7fcb1be00640 0 client.4659 ms_handle_reset on v2:172.21.15.187:6835/3441543802 2024-04-15T15:24:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.483+0000 7fcb40982740 1 -- 172.21.15.150:0/2013449819 shutdown_connections 2024-04-15T15:24:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.483+0000 7fcb40982740 1 -- 172.21.15.150:0/2013449819 wait complete. 2024-04-15T15:24:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.483+0000 7fcb40982740 20 client.4659 trim_cache size 0 max 16384 2024-04-15T15:24:12.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.502+0000 7fcb40982740 1 Processor -- start 2024-04-15T15:24:12.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.502+0000 7fcb40982740 1 -- start start 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.503+0000 7fcb40982740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 0x556876fe7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.503+0000 7fcb40982740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556877035270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.503+0000 7fcb40982740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.503+0000 7fcb40982740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381ff480 con 0x556877036800 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.503+0000 7fcb40982740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381fe480 con 0x5568770195a0 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.503+0000 7fcb40982740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb381f5420 con 0x556877019980 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770357b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51214/0 (socket says 172.21.15.150:51214) 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 -- 172.21.15.150:0/1975232019 learned_addr learned my addr 172.21.15.150:0/1975232019 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 -- 172.21.15.150:0/1975232019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 msgr2=0x556877035270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 --2- 172.21.15.150:0/1975232019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556877035270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 -- 172.21.15.150:0/1975232019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 msgr2=0x556876fe7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 --2- 172.21.15.150:0/1975232019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 0x556876fe7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 -- 172.21.15.150:0/1975232019 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556876f59e60 con 0x556877036800 2024-04-15T15:24:12.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb37400640 1 --2- 172.21.15.150:0/1975232019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770357b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcb30006fe0 tx=0x7fcb30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5851b68b58098804 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:12.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb26a00640 1 -- 172.21.15.150:0/1975232019 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000d040 con 0x556877036800 2024-04-15T15:24:12.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb26a00640 1 -- 172.21.15.150:0/1975232019 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3000f8f0 con 0x556877036800 2024-04-15T15:24:12.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.504+0000 7fcb26a00640 1 -- 172.21.15.150:0/1975232019 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000fce0 con 0x556877036800 2024-04-15T15:24:12.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 -- 172.21.15.150:0/1975232019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 msgr2=0x5568770357b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 --2- 172.21.15.150:0/1975232019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770357b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcb30006fe0 tx=0x7fcb30012000 comp rx=0 tx=0).stop 2024-04-15T15:24:12.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 -- 172.21.15.150:0/1975232019 shutdown_connections 2024-04-15T15:24:12.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 --2- 172.21.15.150:0/1975232019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556877036800 0x5568770357b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 --2- 172.21.15.150:0/1975232019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877019980 0x556877035270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 --2- 172.21.15.150:0/1975232019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5568770195a0 0x556876fe7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 -- 172.21.15.150:0/1975232019 >> 172.21.15.150:0/1975232019 conn(0x556876ff27c0 msgr2=0x556876ff3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:12.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 -- 172.21.15.150:0/1975232019 shutdown_connections 2024-04-15T15:24:12.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.505+0000 7fcb40982740 1 -- 172.21.15.150:0/1975232019 wait complete. 2024-04-15T15:24:12.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.506+0000 7fcb40982740 1 -- start start 2024-04-15T15:24:12.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.506+0000 7fcb40982740 1 Processor -- start 2024-04-15T15:24:12.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb40982740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb40982740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556877019980 0x55687705b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb40982740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877036800 0x556877035270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb40982740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556877035d20 con 0x5568770195a0 2024-04-15T15:24:12.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb40982740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381fdf30 con 0x556877019980 2024-04-15T15:24:12.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb40982740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556877032870 con 0x556877036800 2024-04-15T15:24:12.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb40982740 10 client.0 _ll_register_callbacks cb 0x7fcb30007270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:24:12.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb36a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb36a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51230/0 (socket says 172.21.15.150:51230) 2024-04-15T15:24:12.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb36a00640 1 -- 172.21.15.150:0/1176749062 learned_addr learned my addr 172.21.15.150:0/1176749062 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:12.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb36a00640 1 -- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877036800 msgr2=0x556877035270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.507+0000 7fcb36a00640 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877036800 0x556877035270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb36000640 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556877019980 0x55687705b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb36a00640 1 -- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556877019980 msgr2=0x55687705b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:12.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb36a00640 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556877019980 0x55687705b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb36a00640 1 -- 172.21.15.150:0/1176749062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556877019e40 con 0x5568770195a0 2024-04-15T15:24:12.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb36a00640 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770b4690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb28058510 tx=0x7fcb2808a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d59b0b32171485 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:12.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb1be00640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:24:12.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb1be00640 1 -- 172.21.15.150:0/1176749062 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28022040 con 0x5568770195a0 2024-04-15T15:24:12.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb1be00640 1 -- 172.21.15.150:0/1176749062 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcb2808a240 con 0x5568770195a0 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb40982740 1 -- 172.21.15.150:0/1176749062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3000f8f0 con 0x5568770195a0 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb1be00640 1 -- 172.21.15.150:0/1176749062 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fcb2807e050 con 0x5568770195a0 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb40982740 10 client.4665 fetch_fsmap learned FSMap version 8 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb40982740 1 -- 172.21.15.150:0/1176749062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55687705bba0 con 0x5568770195a0 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.508+0000 7fcb1be00640 1 -- 172.21.15.150:0/1176749062 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3000f8f0 con 0x5568770195a0 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.509+0000 7fcb1be00640 1 -- 172.21.15.150:0/1176749062 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fcb28081080 con 0x5568770195a0 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.509+0000 7fcb40982740 10 client.4665 fetch_fsmap finished waiting for FSMap version 8 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.509+0000 7fcb40982740 10 client.4665 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.509+0000 7fcb40982740 20 client.4665 populate_metadata read hostname 'smithi150' 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.509+0000 7fcb40982740 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] conn(0x55687703a580 0x556877017c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.509+0000 7fcb40982740 4 client.4665 mds_command: new command op to 4250 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.509+0000 7fcb40982740 1 -- 172.21.15.150:0/1176749062 --> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] -- command(tid 0: {"prefix": "status"}) -- 0x556876f5f250 con 0x55687703a580 2024-04-15T15:24:12.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.509+0000 7fcb36000640 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] conn(0x55687703a580 0x556877017c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:12.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.510+0000 7fcb36000640 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] conn(0x55687703a580 0x556877017c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c353482017b5f6 server_cookie=281b9c72b90a7d09 in_seq=0 out_seq=0 2024-04-15T15:24:12.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.510+0000 7fcb1be00640 10 client.4665 ms_handle_connect on v2:172.21.15.187:6835/3441543802 2024-04-15T15:24:12.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb1be00640 1 -- 172.21.15.150:0/1176749062 <== mds.0 v2:172.21.15.187:6835/3441543802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x556876f5f250 con 0x55687703a580 2024-04-15T15:24:12.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb1be00640 10 client.4665 handle_command_reply: tid=0 2024-04-15T15:24:12.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 client.4665 shutdown 2024-04-15T15:24:12.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 -- 172.21.15.150:0/1176749062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 msgr2=0x5568770b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770b4690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb28058510 tx=0x7fcb2808a5e0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 -- 172.21.15.150:0/1176749062 shutdown_connections 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556877036800 0x556877035270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556877019980 0x55687705b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.187:6835/3441543802,v1:172.21.15.187:6837/3441543802] conn(0x55687703a580 0x556877017c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 --2- 172.21.15.150:0/1176749062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568770195a0 0x5568770b4690 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 -- 172.21.15.150:0/1176749062 >> 172.21.15.150:0/1176749062 conn(0x556876ff27c0 msgr2=0x55687704b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb1be00640 0 client.4665 ms_handle_reset on v2:172.21.15.187:6835/3441543802 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.511+0000 7fcb40982740 1 -- 172.21.15.150:0/1176749062 shutdown_connections 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb40982740 1 -- 172.21.15.150:0/1176749062 wait complete. 2024-04-15T15:24:12.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb40982740 20 client.4665 trim_cache size 0 max 16384 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4", 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 4250, 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 17.245681654999998, 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 8, 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 22, 2024-04-15T15:24:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 22, 2024-04-15T15:24:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 22.375898581000001 2024-04-15T15:24:12.521 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 -- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb1406c920 msgr2=0x7fcb1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb1406c920 0x7fcb1406ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb20033000 tx=0x7fcb20009030 comp rx=0 tx=0).stop 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 -- 172.21.15.150:0/1141818432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 msgr2=0x7fcb3813b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb3813b3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb28011a00 tx=0x7fcb28011ec0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 -- 172.21.15.150:0/1141818432 shutdown_connections 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb38159030 0x7fcb381594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb1406c920 0x7fcb1406ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb381f5bc0 0x7fcb381f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 --2- 172.21.15.150:0/1141818432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb1c004800 0x7fcb3813b3f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 -- 172.21.15.150:0/1141818432 >> 172.21.15.150:0/1141818432 conn(0x7fcb380b2460 msgr2=0x7fcb3813eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.512+0000 7fcb3dc00640 1 -- 172.21.15.150:0/1141818432 shutdown_connections 2024-04-15T15:24:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:12.513+0000 7fcb3dc00640 1 -- 172.21.15.150:0/1141818432 wait complete. 2024-04-15T15:24:12.534 INFO:teuthology.run_tasks:Running task sequential... 2024-04-15T15:24:12.546 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-15T15:24:12.546 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-15T15:24:12.546 INFO:teuthology.task.print:Enabling mgr modules 2024-04-15T15:24:12.547 INFO:teuthology.run_tasks:Running task kclient... 2024-04-15T15:24:12.580 INFO:tasks.kclient:Mounting kernel clients... 2024-04-15T15:24:12.580 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-15T15:24:12.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:12.580 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:24:12.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:12.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-04-15T15:24:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-04-15T15:24:12.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:24:12.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:12.738 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:24:12.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:12.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-04-15T15:24:12.875 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-04-15T15:24:12.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:24:12.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:12.879 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:24:12.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:12.937 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-04-15T15:24:13.036 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-04-15T15:24:13.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:24:13.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:13.040 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:24:13.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:13.095 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-04-15T15:24:13.187 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-04-15T15:24:13.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:24:13.191 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-15T15:24:13.191 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:24:13.191 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:24:13.191 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:24:13.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.405+0000 7f6a06000640 1 Processor -- start 2024-04-15T15:24:13.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.405+0000 7f6a06000640 1 -- start start 2024-04-15T15:24:13.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.406+0000 7f6a06000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6a000b51f0 0x7f6a0015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:13.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.406+0000 7f6a06000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0006f940 con 0x7f6a00070210 2024-04-15T15:24:13.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.406+0000 7f6a06000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0006e8a0 con 0x7f6a0015b6d0 2024-04-15T15:24:13.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.406+0000 7f6a06000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0006eb80 con 0x7f6a000b51f0 2024-04-15T15:24:13.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.407+0000 7f69fea00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6a00070210 0x7f6a000b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42286/0 (socket says 172.21.15.150:42286) 2024-04-15T15:24:13.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.407+0000 7f69fea00640 1 -- 172.21.15.150:0/3834491069 learned_addr learned my addr 172.21.15.150:0/3834491069 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:13.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.407+0000 7f69ffe00640 1 --2- 172.21.15.150:0/3834491069 >> v2:172.21.15.150:3300/0 conn(0x7f6a000b51f0 0x7f6a0015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:13.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.407+0000 7f69fe000640 1 -- 172.21.15.150:0/3834491069 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1656365721 0 0) 0x7f6a0006f940 con 0x7f6a00070210 2024-04-15T15:24:13.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.407+0000 7f69fe000640 1 -- 172.21.15.150:0/3834491069 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69e4003520 con 0x7f6a00070210 2024-04-15T15:24:13.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.407+0000 7f69ffe00640 1 -- 172.21.15.150:0/3834491069 >> v1:172.21.15.187:6789/0 conn(0x7f6a0015b6d0 legacy=0x7f6a000cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:13.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.407+0000 7f69ffe00640 1 -- 172.21.15.150:0/3834491069 >> v1:172.21.15.150:6789/0 conn(0x7f6a00070210 legacy=0x7f6a000b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:13.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.407+0000 7f69ffe00640 1 -- 172.21.15.150:0/3834491069 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0006ee60 con 0x7f6a000b51f0 2024-04-15T15:24:13.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.408+0000 7f69ffe00640 1 --2- 172.21.15.150:0/3834491069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a0015cac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009d90 tx=0x7f69f802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a6ca5563c3ff05 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:13.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f69fe000640 1 -- 172.21.15.150:0/3834491069 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8039070 con 0x7f6a000b51f0 2024-04-15T15:24:13.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f69fe000640 1 -- 172.21.15.150:0/3834491069 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f802cc60 con 0x7f6a000b51f0 2024-04-15T15:24:13.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f69fe000640 1 -- 172.21.15.150:0/3834491069 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8034640 con 0x7f6a000b51f0 2024-04-15T15:24:13.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f6a06000640 1 -- 172.21.15.150:0/3834491069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 msgr2=0x7f6a0015cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:13.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f6a06000640 1 --2- 172.21.15.150:0/3834491069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a0015cac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009d90 tx=0x7f69f802beb0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f6a06000640 1 -- 172.21.15.150:0/3834491069 shutdown_connections 2024-04-15T15:24:13.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f6a06000640 1 --2- 172.21.15.150:0/3834491069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a0015cac0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f6a06000640 1 -- 172.21.15.150:0/3834491069 >> 172.21.15.150:0/3834491069 conn(0x7f6a000b27b0 msgr2=0x7f6a000b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:13.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f6a06000640 1 -- 172.21.15.150:0/3834491069 shutdown_connections 2024-04-15T15:24:13.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.409+0000 7f6a06000640 1 -- 172.21.15.150:0/3834491069 wait complete. 2024-04-15T15:24:13.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.411+0000 7f6a06000640 1 Processor -- start 2024-04-15T15:24:13.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.411+0000 7f6a06000640 1 -- start start 2024-04-15T15:24:13.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f6a06000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a00070210 0x7f6a001506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:13.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f6a06000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a00152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:13.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f6a06000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0015b6d0 0x7f6a000b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:13.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f6a06000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a000ce420 con 0x7f6a000b51f0 2024-04-15T15:24:13.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f6a06000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0006f420 con 0x7f6a0015b6d0 2024-04-15T15:24:13.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f6a06000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6a00166300 con 0x7f6a00070210 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f69fea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a00152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f69fea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a00152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51254/0 (socket says 172.21.15.150:51254) 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.412+0000 7f69fea00640 1 -- 172.21.15.150:0/4259308100 learned_addr learned my addr 172.21.15.150:0/4259308100 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69ffe00640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0015b6d0 0x7f6a000b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69ff400640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a00070210 0x7f6a001506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69fea00640 1 -- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a00070210 msgr2=0x7f6a001506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69fea00640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a00070210 0x7f6a001506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69fea00640 1 -- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0015b6d0 msgr2=0x7f6a000b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69fea00640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0015b6d0 0x7f6a000b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69fea00640 1 -- 172.21.15.150:0/4259308100 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a00153020 con 0x7f6a000b51f0 2024-04-15T15:24:13.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69ff400640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a00070210 0x7f6a001506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:13.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69fea00640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a00152660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00c530 tx=0x7f69ec00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:13.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69f7400640 1 -- 172.21.15.150:0/4259308100 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec017070 con 0x7f6a000b51f0 2024-04-15T15:24:13.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69f7400640 1 -- 172.21.15.150:0/4259308100 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ec005ce0 con 0x7f6a000b51f0 2024-04-15T15:24:13.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a00153300 con 0x7f6a000b51f0 2024-04-15T15:24:13.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.413+0000 7f69f7400640 1 -- 172.21.15.150:0/4259308100 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0072b0 con 0x7f6a000b51f0 2024-04-15T15:24:13.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.414+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a001589a0 con 0x7f6a000b51f0 2024-04-15T15:24:13.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.415+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a00068c80 con 0x7f6a000b51f0 2024-04-15T15:24:13.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.415+0000 7f69f7400640 1 -- 172.21.15.150:0/4259308100 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69ec01a070 con 0x7f6a000b51f0 2024-04-15T15:24:13.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.416+0000 7f69f7400640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f69e006c8f0 0x7f69e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:13.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.416+0000 7f69f7400640 1 -- 172.21.15.150:0/4259308100 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f69ec08afe0 con 0x7f6a000b51f0 2024-04-15T15:24:13.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.417+0000 7f69ff400640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f69e006c8f0 0x7f69e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:13.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.418+0000 7f69ff400640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f69e006c8f0 0x7f69e006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f69f0000f60 tx=0x7f69f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:13.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.421+0000 7f69f7400640 1 -- 172.21.15.150:0/4259308100 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69ec0594e0 con 0x7f6a000b51f0 2024-04-15T15:24:13.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.662+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6a000d0560 con 0x7f6a000b51f0 2024-04-15T15:24:13.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.663+0000 7f69f7400640 1 -- 172.21.15.150:0/4259308100 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f69ec058de0 con 0x7f6a000b51f0 2024-04-15T15:24:13.665 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:24:13.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.665+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f69e006c8f0 msgr2=0x7f69e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:13.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f69e006c8f0 0x7f69e006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f69f0000f60 tx=0x7f69f0009040 comp rx=0 tx=0).stop 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 msgr2=0x7f6a00152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a00152660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00c530 tx=0x7f69ec00c9f0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 shutdown_connections 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0015b6d0 0x7f6a000b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a000b51f0 0x7f6a00152660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f69e006c8f0 0x7f69e006edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 --2- 172.21.15.150:0/4259308100 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a00070210 0x7f6a001506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 >> 172.21.15.150:0/4259308100 conn(0x7f6a000b27b0 msgr2=0x7f6a000aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 shutdown_connections 2024-04-15T15:24:13.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13.666+0000 7f6a06000640 1 -- 172.21.15.150:0/4259308100 wait complete. 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:24:13.686 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:24:13.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:13.687 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-04-15T15:24:13.735 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:24:13.735 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:13.736 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:13.737 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:24:13.737 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:13.737 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:24:13.737 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42561sec preferred_lft 42561sec 2024-04-15T15:24:13.737 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2024-04-15T15:24:13.737 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:13.738 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-15T15:24:13.738 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:13.738 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:13.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2024-04-15T15:24:13.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2024-04-15T15:24:13.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2024-04-15T15:24:13.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-15T15:24:13.738 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:13.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:14.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:14.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:14.255 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:24:14.332 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-15T15:24:14.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:14.337 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-04-15T15:24:14.397 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-04-15T15:24:14.397 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:24:14.397 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:24:14.397 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:24:14.397 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:24:14.397 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:14.398 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:14.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:24:14.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:24:14.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:24:14.398 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:14.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:15.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:15.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:15.259 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:24:15.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:15.313 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-04-15T15:24:15.366 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:15.366 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:15.366 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:24:15.366 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-15T15:24:15.366 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:15.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:15.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:15.578 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-15T15:24:15.579 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:15.579 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:15.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-15T15:24:15.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-15T15:24:15.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-15T15:24:15.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-15T15:24:15.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-15T15:24:15.579 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:15.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:15.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:15.910 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:15.910 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:15.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-04-15T15:24:15.911 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2024-04-15T15:24:15.911 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:16.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:16.128 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:24:16.129 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:24:16.173 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:24:16.173 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:24:16.173 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:24:16.268 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:24:16.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:16.268 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:24:16.393 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:24:16.393 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:24:16.394 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:24:16.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:24:16.516 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:24:17.087 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:24:17.087 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:24:17.087 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:24:17.087 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:24:17.087 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:24:17.123 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:24:17.124 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:24:17.124 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:24:17.124 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:24:17.124 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:24:17.124 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:24:17.124 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-04-15T15:24:17.124 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:24:17.126 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-15T15:24:17.126 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:24:17.126 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:24:17.126 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:24:17.127 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:24:17.131 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:24:17.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:17.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:24:17.224 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:17.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:17.373 INFO:teuthology.orchestra.run.smithi150.stdout:client4409 2024-04-15T15:24:17.373 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:24:17.373 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:17.425 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:17.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:17.566 INFO:teuthology.orchestra.run.smithi150.stdout:client4409 2024-04-15T15:24:17.566 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:17.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:17.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:17.703 INFO:teuthology.orchestra.run.smithi150.stdout:client4409 2024-04-15T15:24:17.704 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4.client4409/status') 2024-04-15T15:24:17.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:17.868 INFO:teuthology.orchestra.run.smithi150.stdout:instance: client.4409 (3)172.21.15.150:0/3304110628 2024-04-15T15:24:17.868 INFO:teuthology.orchestra.run.smithi150.stdout:blocklisted: false 2024-04-15T15:24:17.868 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records in 2024-04-15T15:24:17.868 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records out 2024-04-15T15:24:17.868 INFO:teuthology.orchestra.run.smithi150.stderr:71 bytes copied, 7.2983e-05 s, 973 kB/s 2024-04-15T15:24:17.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:17.883 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-15T15:24:17.883 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:24:17.883 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:24:17.883 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:24:18.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.089+0000 7f861b200640 1 Processor -- start 2024-04-15T15:24:18.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.089+0000 7f861b200640 1 -- start start 2024-04-15T15:24:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.090+0000 7f861b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614070210 0x7f86140b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.091+0000 7f861b200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861406f940 con 0x7f86140b5020 2024-04-15T15:24:18.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.091+0000 7f861b200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861406e8a0 con 0x7f861415b6d0 2024-04-15T15:24:18.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.091+0000 7f861b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86140b5480 con 0x7f8614070210 2024-04-15T15:24:18.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.091+0000 7f8619200640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f86140b5020 0x7f861415bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:36016/0 (socket says 172.21.15.150:36016) 2024-04-15T15:24:18.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.091+0000 7f8619200640 1 -- 172.21.15.150:0/2279161101 learned_addr learned my addr 172.21.15.150:0/2279161101 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.091+0000 7f8613400640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614070210 0x7f86140b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.091+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2314563902 0 0) 0x7f861406f940 con 0x7f86140b5020 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.091+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85f8003520 con 0x7f86140b5020 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 122745240 0 0) 0x7f85f8003520 con 0x7f86140b5020 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614070210 msgr2=0x7f86140b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614070210 0x7f86140b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 >> v1:172.21.15.187:6789/0 conn(0x7f861415b6d0 legacy=0x7f86140cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861406eb80 con 0x7f86140b5020 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f860c0019f0 con 0x7f86140b5020 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 >> v1:172.21.15.150:6789/0 conn(0x7f86140b5020 legacy=0x7f861415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:18.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85f80039e0 0x7f85f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f8004800 0x7f85f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614070210 0x7f85f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85f800baf0 con 0x7f85f80039e0 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85f800bca0 con 0x7f85f8004800 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f85f800e0b0 con 0x7f8614070210 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.092+0000 7f8613e00640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85f80039e0 0x7f85f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.093+0000 7f8619200640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614070210 0x7f85f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.093+0000 7f8613400640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f8004800 0x7f85f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.093+0000 7f8613e00640 1 -- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614070210 msgr2=0x7f85f8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.093+0000 7f8613e00640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614070210 0x7f85f8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.093+0000 7f8613e00640 1 -- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f8004800 msgr2=0x7f85f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.093+0000 7f8613e00640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f8004800 0x7f85f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.093+0000 7f8613e00640 1 -- 172.21.15.150:0/2279161101 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f8003520 con 0x7f85f80039e0 2024-04-15T15:24:18.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.093+0000 7f8613400640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f8004800 0x7f85f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.094+0000 7f8613e00640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85f80039e0 0x7f85f8003e00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8604001750 tx=0x7f85f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78617e4b2fd0e80d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.094+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604001370 con 0x7f85f80039e0 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.094+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86040052d0 con 0x7f85f80039e0 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.094+0000 7f8612a00640 1 -- 172.21.15.150:0/2279161101 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604003830 con 0x7f85f80039e0 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.095+0000 7f861b200640 1 -- 172.21.15.150:0/2279161101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85f80039e0 msgr2=0x7f85f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.095+0000 7f861b200640 1 --2- 172.21.15.150:0/2279161101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85f80039e0 0x7f85f8003e00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8604001750 tx=0x7f85f8005830 comp rx=0 tx=0).stop 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.095+0000 7f8619200640 1 -- 172.21.15.150:0/2279161101 reap_dead start 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.095+0000 7f861b200640 1 -- 172.21.15.150:0/2279161101 shutdown_connections 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.095+0000 7f861b200640 1 -- 172.21.15.150:0/2279161101 >> 172.21.15.150:0/2279161101 conn(0x7f86140b27b0 msgr2=0x7f86140b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.095+0000 7f861b200640 1 -- 172.21.15.150:0/2279161101 shutdown_connections 2024-04-15T15:24:18.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.095+0000 7f861b200640 1 -- 172.21.15.150:0/2279161101 wait complete. 2024-04-15T15:24:18.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.097+0000 7f861b200640 1 Processor -- start 2024-04-15T15:24:18.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.097+0000 7f861b200640 1 -- start start 2024-04-15T15:24:18.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f861b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f80039e0 0x7f8614150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:18.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f861b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614155090 0x7f8614155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f861b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86141e3860 0x7f86141e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f861b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f861406f420 con 0x7f86141e3860 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f861b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f861406f180 con 0x7f85f80039e0 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f861b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86140ce420 con 0x7f8614155090 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f8619200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86141e3860 0x7f86141e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f8619200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86141e3860 0x7f86141e3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48714/0 (socket says 172.21.15.150:48714) 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.098+0000 7f8619200640 1 -- 172.21.15.150:0/1168834609 learned_addr learned my addr 172.21.15.150:0/1168834609 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.099+0000 7f8613400640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614155090 0x7f8614155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.099+0000 7f8613e00640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f80039e0 0x7f8614150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.099+0000 7f8619200640 1 -- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614155090 msgr2=0x7f8614155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.099+0000 7f8619200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614155090 0x7f8614155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.099+0000 7f8619200640 1 -- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f80039e0 msgr2=0x7f8614150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.099+0000 7f8619200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f80039e0 0x7f8614150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.099+0000 7f8619200640 1 -- 172.21.15.150:0/1168834609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86141f0710 con 0x7f86141e3860 2024-04-15T15:24:18.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.099+0000 7f8619200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86141e3860 0x7f86141e3d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86140cebd0 tx=0x7f860c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:18.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.100+0000 7f860be00640 1 -- 172.21.15.150:0/1168834609 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c00ea40 con 0x7f86141e3860 2024-04-15T15:24:18.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.100+0000 7f860be00640 1 -- 172.21.15.150:0/1168834609 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c018070 con 0x7f86141e3860 2024-04-15T15:24:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.100+0000 7f860be00640 1 -- 172.21.15.150:0/1168834609 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c00ee20 con 0x7f86141e3860 2024-04-15T15:24:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.100+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86141e4250 con 0x7f86141e3860 2024-04-15T15:24:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.100+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86141f0c90 con 0x7f86141e3860 2024-04-15T15:24:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.101+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8614068c80 con 0x7f86141e3860 2024-04-15T15:24:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.102+0000 7f860be00640 1 -- 172.21.15.150:0/1168834609 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f860c01d460 con 0x7f86141e3860 2024-04-15T15:24:18.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.103+0000 7f860be00640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f85f406c8f0 0x7f85f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:18.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.103+0000 7f860be00640 1 -- 172.21.15.150:0/1168834609 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f860c08bdd0 con 0x7f86141e3860 2024-04-15T15:24:18.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.103+0000 7f8613e00640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f85f406c8f0 0x7f85f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:18.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.104+0000 7f8613e00640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f85f406c8f0 0x7f85f406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8604033000 tx=0x7f8604009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.108+0000 7f860be00640 1 -- 172.21.15.150:0/1168834609 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f860c05a2b0 con 0x7f86141e3860 2024-04-15T15:24:18.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.352+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f86140cdfb0 con 0x7f86141e3860 2024-04-15T15:24:18.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.353+0000 7f860be00640 1 -- 172.21.15.150:0/1168834609 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f860c059bb0 con 0x7f86141e3860 2024-04-15T15:24:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:24:18.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.355+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f85f406c8f0 msgr2=0x7f85f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:18.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.355+0000 7f861b200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f85f406c8f0 0x7f85f406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8604033000 tx=0x7f8604009030 comp rx=0 tx=0).stop 2024-04-15T15:24:18.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.355+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86141e3860 msgr2=0x7f86141e3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:18.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.355+0000 7f861b200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86141e3860 0x7f86141e3d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86140cebd0 tx=0x7f860c007b00 comp rx=0 tx=0).stop 2024-04-15T15:24:18.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.356+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 shutdown_connections 2024-04-15T15:24:18.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.356+0000 7f861b200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86141e3860 0x7f86141e3d10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.356+0000 7f861b200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f85f406c8f0 0x7f85f406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.356+0000 7f861b200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8614155090 0x7f8614155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.356+0000 7f861b200640 1 --2- 172.21.15.150:0/1168834609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85f80039e0 0x7f8614150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:18.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.356+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 >> 172.21.15.150:0/1168834609 conn(0x7f86140b27b0 msgr2=0x7f86140b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:18.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.356+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 shutdown_connections 2024-04-15T15:24:18.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18.356+0000 7f861b200640 1 -- 172.21.15.150:0/1168834609 wait complete. 2024-04-15T15:24:18.377 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:24:18.377 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:24:18.377 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-04-15T15:24:18.377 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-15T15:24:18.377 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:24:18.378 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:24:18.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:24:18.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:24:18.378 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:24:18.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:18.378 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-04-15T15:24:18.430 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:24:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:24:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:24:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-15T15:24:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:18.430 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42556sec preferred_lft 42556sec 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:18.431 INFO:teuthology.orchestra.run.smithi150.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 76:7b:44:2c:2d:50 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::1086:faff:fe4c:731a/64 scope link 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 76:7b:44:2c:2d:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::747b:44ff:fe2c:2d50/64 scope link 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:18.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:18.432 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:24:18.516 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-15T15:24:18.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:18.521 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-04-15T15:24:18.573 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-04-15T15:24:18.573 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:24:18.573 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:24:18.573 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:24:18.574 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:24:18.575 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:18.575 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:18.575 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:24:18.575 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:24:18.575 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:24:18.575 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:18.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:18.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:18.827 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:24:18.877 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:24:18.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:18.878 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-04-15T15:24:18.927 INFO:teuthology.orchestra.run.smithi150.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-15T15:24:18.928 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:18.928 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:18.928 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:24:18.928 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-15T15:24:18.928 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:19.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:19.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:19.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:19.127 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether c6:87:b2:e1:31:93 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:19.219 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::c487:b2ff:fee1:3193/64 scope link 2024-04-15T15:24:19.220 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:19.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:19.224 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-15T15:24:19.323 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether c6:87:b2:e1:31:93 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::c487:b2ff:fee1:3193/64 scope link 2024-04-15T15:24:19.324 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:19.328 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-15T15:24:19.328 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:19.328 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:19.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-15T15:24:19.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-15T15:24:19.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-15T15:24:19.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-15T15:24:19.329 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-15T15:24:19.329 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:19.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:19.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:19.641 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:19.641 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:24:19.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.1 up 2024-04-15T15:24:19.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.1 master ceph-brx 2024-04-15T15:24:19.641 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:24:19.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:19.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:19.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:24:19.865 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-15T15:24:19.911 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:24:19.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:24:19.911 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:24:20.005 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:24:20.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:20.005 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:24:20.136 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:24:20.136 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:24:20.136 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:24:20.158 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:24:20.158 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:24:20.158 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:24:20.158 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:24:20.158 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:24:20.186 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:24:20.186 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:24:20.186 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:24:20.186 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:24:20.187 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:24:20.187 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:24:20.187 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-04-15T15:24:20.187 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:24:20.188 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-15T15:24:20.188 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:24:20.188 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:24:20.188 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:24:20.188 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:24:20.192 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:24:20.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:20.192 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:24:20.275 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:20.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:20.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:20.394 INFO:teuthology.orchestra.run.smithi150.stdout:client4683 2024-04-15T15:24:20.394 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:24:20.394 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:20.445 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:20.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:20.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:20.563 INFO:teuthology.orchestra.run.smithi150.stdout:client4683 2024-04-15T15:24:20.564 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:20.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:20.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:20.681 INFO:teuthology.orchestra.run.smithi150.stdout:client4683 2024-04-15T15:24:20.681 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4.client4683/status') 2024-04-15T15:24:20.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:20.827 INFO:teuthology.orchestra.run.smithi150.stdout:instance: client.4683 (3)192.168.0.2:0/145058014 2024-04-15T15:24:20.827 INFO:teuthology.orchestra.run.smithi150.stdout:blocklisted: false 2024-04-15T15:24:20.827 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records in 2024-04-15T15:24:20.827 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records out 2024-04-15T15:24:20.827 INFO:teuthology.orchestra.run.smithi150.stderr:68 bytes copied, 6.3379e-05 s, 1.1 MB/s 2024-04-15T15:24:20.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:20.840 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-15T15:24:20.840 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:24:20.840 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:24:20.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:24:21.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.046+0000 7fadf6c00640 1 Processor -- start 2024-04-15T15:24:21.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.047+0000 7fadf6c00640 1 -- start start 2024-04-15T15:24:21.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.048+0000 7fadf6c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf01461f0 0x7fadf00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.048+0000 7fadf6c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fadf00b3fe0 0x7fadf015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.048+0000 7fadf6c00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf00b4be0 con 0x7fadf00cedf0 2024-04-15T15:24:21.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.048+0000 7fadf6c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fadf00a9640 con 0x7fadf00b3fe0 2024-04-15T15:24:21.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.048+0000 7fadf6c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadf00cf250 con 0x7fadf01461f0 2024-04-15T15:24:21.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fade7400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fadf00cedf0 0x7fadf00b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:36032/0 (socket says 172.21.15.150:36032) 2024-04-15T15:24:21.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fade7400640 1 -- 172.21.15.150:0/4179502721 learned_addr learned my addr 172.21.15.150:0/4179502721 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:21.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadf4c00640 1 --2- 172.21.15.150:0/4179502721 >> v2:172.21.15.187:3300/0 conn(0x7fadf00b3fe0 0x7fadf015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadefe00640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf01461f0 0x7fadf00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157318516 0 0) 0x7fadf00b4be0 con 0x7fadf00cedf0 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadd4003520 con 0x7fadf00cedf0 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 961920814 0 0) 0x7fadd4003520 con 0x7fadf00cedf0 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf01461f0 msgr2=0x7fadf00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf01461f0 0x7fadf00ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 >> v2:172.21.15.187:3300/0 conn(0x7fadf00b3fe0 msgr2=0x7fadf015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 --2- 172.21.15.150:0/4179502721 >> v2:172.21.15.187:3300/0 conn(0x7fadf00b3fe0 0x7fadf015d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf00b4620 con 0x7fadf00cedf0 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fadd8001670 con 0x7fadf00cedf0 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.049+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 >> v1:172.21.15.150:6789/0 conn(0x7fadf00cedf0 legacy=0x7fadf00b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadef400640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadd40039b0 0x7fadd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadef400640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf01461f0 0x7fadd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadef400640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd4004d40 0x7fadd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadd400b9c0 con 0x7fadd4004d40 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadd400bb40 con 0x7fadd40039b0 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadd400bcc0 con 0x7fadf01461f0 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadf4c00640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd4004d40 0x7fadd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fade7400640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf01461f0 0x7fadd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadefe00640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadd40039b0 0x7fadd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadf4c00640 1 -- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf01461f0 msgr2=0x7fadd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadf4c00640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf01461f0 0x7fadd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadf4c00640 1 -- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadd40039b0 msgr2=0x7fadd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadf4c00640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadd40039b0 0x7fadd4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.050+0000 7fadf4c00640 1 -- 172.21.15.150:0/4179502721 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd4003520 con 0x7fadd4004d40 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadf4c00640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd4004d40 0x7fadd40051c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fade8009fc0 tx=0x7fadd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca9dc96b4fe7b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8039070 con 0x7fadd4004d40 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade802cc60 con 0x7fadd4004d40 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadef400640 1 -- 172.21.15.150:0/4179502721 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8009c10 con 0x7fadd4004d40 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadf6c00640 1 -- 172.21.15.150:0/4179502721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd4004d40 msgr2=0x7fadd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadf6c00640 1 --2- 172.21.15.150:0/4179502721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd4004d40 0x7fadd40051c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fade8009fc0 tx=0x7fadd4005830 comp rx=0 tx=0).stop 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadf4c00640 1 -- 172.21.15.150:0/4179502721 reap_dead start 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadf6c00640 1 -- 172.21.15.150:0/4179502721 shutdown_connections 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.052+0000 7fadf6c00640 1 -- 172.21.15.150:0/4179502721 >> 172.21.15.150:0/4179502721 conn(0x7fadf00b27b0 msgr2=0x7fadf00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:21.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.053+0000 7fadf6c00640 1 -- 172.21.15.150:0/4179502721 shutdown_connections 2024-04-15T15:24:21.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.053+0000 7fadf6c00640 1 -- 172.21.15.150:0/4179502721 wait complete. 2024-04-15T15:24:21.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.054+0000 7fadf6c00640 1 Processor -- start 2024-04-15T15:24:21.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.054+0000 7fadf6c00640 1 -- start start 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.055+0000 7fadf6c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadd4004d40 0x7fadf01f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.055+0000 7fadf6c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadf00b0ee0 0x7fadf015c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.055+0000 7fadf6c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf00ae350 0x7fadf00ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.055+0000 7fadf6c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadf015d290 con 0x7fadf00b0ee0 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.055+0000 7fadf6c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadf015cfb0 con 0x7fadf00ae350 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.055+0000 7fadf6c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadf01465d0 con 0x7fadd4004d40 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadf00b0ee0 0x7fadf015c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadf00b0ee0 0x7fadf015c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48726/0 (socket says 172.21.15.150:48726) 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 -- 172.21.15.150:0/2213212564 learned_addr learned my addr 172.21.15.150:0/2213212564 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fadefe00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadd4004d40 0x7fadf01f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 -- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadd4004d40 msgr2=0x7fadf01f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fadf4c00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf00ae350 0x7fadf00ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadd4004d40 0x7fadf01f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 -- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf00ae350 msgr2=0x7fadf00ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf00ae350 0x7fadf00ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 -- 172.21.15.150:0/2213212564 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf00aed10 con 0x7fadf00b0ee0 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.056+0000 7fade7400640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadf00b0ee0 0x7fadf015c2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e870 tx=0x7fadd800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.057+0000 7fadecc00640 1 -- 172.21.15.150:0/2213212564 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd800c8d0 con 0x7fadf00b0ee0 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.057+0000 7fadecc00640 1 -- 172.21.15.150:0/2213212564 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd8018070 con 0x7fadf00b0ee0 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.057+0000 7fadecc00640 1 -- 172.21.15.150:0/2213212564 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd800ccd0 con 0x7fadf00b0ee0 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.057+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf00b16c0 con 0x7fadf00b0ee0 2024-04-15T15:24:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.057+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf00a9750 con 0x7fadf00b0ee0 2024-04-15T15:24:21.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.058+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf0068250 con 0x7fadf00b0ee0 2024-04-15T15:24:21.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.059+0000 7fadecc00640 1 -- 172.21.15.150:0/2213212564 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadd801d460 con 0x7fadf00b0ee0 2024-04-15T15:24:21.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.060+0000 7fadecc00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fadcc06c920 0x7fadcc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:21.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.060+0000 7fadecc00640 1 -- 172.21.15.150:0/2213212564 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fadd808bb10 con 0x7fadf00b0ee0 2024-04-15T15:24:21.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.061+0000 7fadefe00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fadcc06c920 0x7fadcc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:21.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.062+0000 7fadefe00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fadcc06c920 0x7fadcc06ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fadf00adf50 tx=0x7fade0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:21.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.065+0000 7fadecc00640 1 -- 172.21.15.150:0/2213212564 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fadf0068250 con 0x7fadf00b0ee0 2024-04-15T15:24:21.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.306+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fadf0068c80 con 0x7fadf00b0ee0 2024-04-15T15:24:21.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.307+0000 7fadecc00640 1 -- 172.21.15.150:0/2213212564 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fadf00aed10 con 0x7fadf00b0ee0 2024-04-15T15:24:21.309 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:24:21.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fadcc06c920 msgr2=0x7fadcc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fadcc06c920 0x7fadcc06ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fadf00adf50 tx=0x7fade0008040 comp rx=0 tx=0).stop 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadf00b0ee0 msgr2=0x7fadf015c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadf00b0ee0 0x7fadf015c2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e870 tx=0x7fadd800ed30 comp rx=0 tx=0).stop 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 shutdown_connections 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf00ae350 0x7fadf00ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadf00b0ee0 0x7fadf015c2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fadcc06c920 0x7fadcc06ede0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 --2- 172.21.15.150:0/2213212564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadd4004d40 0x7fadf01f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 >> 172.21.15.150:0/2213212564 conn(0x7fadf00b27b0 msgr2=0x7fadf015df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 shutdown_connections 2024-04-15T15:24:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:21.310+0000 7fadf6c00640 1 -- 172.21.15.150:0/2213212564 wait complete. 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:24:21.331 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:24:21.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:21.332 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:21.389 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42601sec preferred_lft 42601sec 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-04-15T15:24:21.390 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:21.391 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-15T15:24:21.391 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:21.391 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:21.392 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add name ceph-brx type bridge 2024-04-15T15:24:21.392 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr flush dev ceph-brx 2024-04-15T15:24:21.392 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set ceph-brx up 2024-04-15T15:24:21.392 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-15T15:24:21.392 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:21.498 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:21.921 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:21.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:21.931 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:24:22.003 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-15T15:24:22.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:22.007 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-04-15T15:24:22.069 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-04-15T15:24:22.069 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:24:22.069 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:24:22.069 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:24:22.069 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:24:22.070 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:22.070 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:22.070 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:24:22.070 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:24:22.070 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:24:22.070 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:22.173 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:22.837 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:22.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:22.847 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:24:22.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:22.900 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-04-15T15:24:22.950 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:22.951 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:22.951 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:24:22.951 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-15T15:24:22.951 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:23.053 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:23.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:23.157 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-15T15:24:23.157 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:23.157 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:23.157 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-15T15:24:23.157 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-15T15:24:23.157 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-15T15:24:23.158 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-15T15:24:23.158 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-15T15:24:23.158 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:23.263 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:23.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:23.502 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:23.502 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:23.502 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-04-15T15:24:23.502 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.0 master ceph-brx 2024-04-15T15:24:23.502 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:23.603 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:23.721 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:23.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:24:23.730 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:24:23.773 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:24:23.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:24:23.773 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:24:23.864 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:24:23.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:23.865 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:24:23.997 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:24:23.997 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:24:23.997 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:24:24.730 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:24:24.730 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:24:24.730 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:24:24.731 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:24:24.731 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:24:24.763 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:24:24.763 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:24:24.763 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:24:24.763 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:24:24.764 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:24:24.764 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:24:24.764 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-04-15T15:24:24.764 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:24:24.766 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-15T15:24:24.766 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:24:24.766 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:24:24.766 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:24:24.766 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:24:24.772 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:24:24.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:24.773 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:24:24.853 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:24.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:24.959 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:24.968 INFO:teuthology.orchestra.run.smithi187.stdout:client4531 2024-04-15T15:24:24.968 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:24:24.968 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:25.021 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:25.123 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:25.130 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:25.139 INFO:teuthology.orchestra.run.smithi187.stdout:client4531 2024-04-15T15:24:25.139 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:25.240 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:25.245 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:25.254 INFO:teuthology.orchestra.run.smithi187.stdout:client4531 2024-04-15T15:24:25.254 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4.client4531/status') 2024-04-15T15:24:25.354 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:25.393 INFO:teuthology.orchestra.run.smithi187.stdout:instance: client.4531 (3)192.168.0.1:0/4283134925 2024-04-15T15:24:25.393 INFO:teuthology.orchestra.run.smithi187.stdout:blocklisted: false 2024-04-15T15:24:25.393 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records in 2024-04-15T15:24:25.393 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records out 2024-04-15T15:24:25.393 INFO:teuthology.orchestra.run.smithi187.stderr:69 bytes copied, 6.3094e-05 s, 1.1 MB/s 2024-04-15T15:24:25.395 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:25.405 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-15T15:24:25.406 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-15T15:24:25.406 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:24:25.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:24:25.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.625+0000 7f480dc00640 1 Processor -- start 2024-04-15T15:24:25.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.626+0000 7f480dc00640 1 -- start start 2024-04-15T15:24:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.627+0000 7f480dc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4808162020 0x7f4808162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.627+0000 7f480dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080cf110 0x7f48080d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.627+0000 7f480dc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48080d0090 con 0x7f48080d1ac0 2024-04-15T15:24:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.627+0000 7f480dc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f48080d1f50 con 0x7f4808162020 2024-04-15T15:24:25.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.627+0000 7f480dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4808163440 con 0x7f48080cf110 2024-04-15T15:24:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.628+0000 7f47ff400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080cf110 0x7f48080d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.628+0000 7f4807e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4808162020 0x7f4808162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.628+0000 7f47ff400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080cf110 0x7f48080d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:44406/0 (socket says 172.21.15.150:44406) 2024-04-15T15:24:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.628+0000 7f47ff400640 1 -- 172.21.15.150:0/1211941029 learned_addr learned my addr 172.21.15.150:0/1211941029 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.628+0000 7f4807e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4808162020 0x7f4808162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:37766/0 (socket says 172.21.15.150:37766) 2024-04-15T15:24:25.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.628+0000 7f480d200640 1 --1- 172.21.15.150:0/1211941029 >> v1:172.21.15.187:6789/0 conn(0x7f48080d1ac0 0x7f48080b62a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:48000/0 (socket says 172.21.15.150:48000) 2024-04-15T15:24:25.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.629+0000 7f47ff400640 1 -- 172.21.15.150:0/1211941029 >> v2:172.21.15.187:3300/0 conn(0x7f4808162020 msgr2=0x7f4808162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:25.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.629+0000 7f47ff400640 1 --2- 172.21.15.150:0/1211941029 >> v2:172.21.15.187:3300/0 conn(0x7f4808162020 0x7f4808162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.629+0000 7f47ff400640 1 -- 172.21.15.150:0/1211941029 >> v1:172.21.15.187:6789/0 conn(0x7f48080d1ac0 legacy=0x7f48080b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:25.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.629+0000 7f47ff400640 1 -- 172.21.15.150:0/1211941029 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48080cfad0 con 0x7f48080cf110 2024-04-15T15:24:25.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.630+0000 7f47ff400640 1 --2- 172.21.15.150:0/1211941029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080cf110 0x7f48080d1580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009870 tx=0x7f47f002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499432679aa2da7d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:25.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f4807400640 1 -- 172.21.15.150:0/1211941029 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0034070 con 0x7f48080cf110 2024-04-15T15:24:25.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f4807400640 1 -- 172.21.15.150:0/1211941029 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f0030840 con 0x7f48080cf110 2024-04-15T15:24:25.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f4807400640 1 -- 172.21.15.150:0/1211941029 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0030c10 con 0x7f48080cf110 2024-04-15T15:24:25.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f480dc00640 1 -- 172.21.15.150:0/1211941029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080cf110 msgr2=0x7f48080d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:25.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f480dc00640 1 --2- 172.21.15.150:0/1211941029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080cf110 0x7f48080d1580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009870 tx=0x7f47f002b980 comp rx=0 tx=0).stop 2024-04-15T15:24:25.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f480dc00640 1 -- 172.21.15.150:0/1211941029 shutdown_connections 2024-04-15T15:24:25.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f480dc00640 1 --2- 172.21.15.150:0/1211941029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080cf110 0x7f48080d1580 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f480dc00640 1 --2- 172.21.15.150:0/1211941029 >> v2:172.21.15.187:3300/0 conn(0x7f4808162020 0x7f4808162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f480dc00640 1 -- 172.21.15.150:0/1211941029 >> 172.21.15.150:0/1211941029 conn(0x7f480806cf00 msgr2=0x7f480806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:25.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f480dc00640 1 -- 172.21.15.150:0/1211941029 shutdown_connections 2024-04-15T15:24:25.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.631+0000 7f480dc00640 1 -- 172.21.15.150:0/1211941029 wait complete. 2024-04-15T15:24:25.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.633+0000 7f480dc00640 1 Processor -- start 2024-04-15T15:24:25.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.633+0000 7f480dc00640 1 -- start start 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.634+0000 7f480dc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48080cf110 0x7f4808157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.634+0000 7f480dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080d1ac0 0x7f48080c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.634+0000 7f480dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4808162020 0x7f48080cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.634+0000 7f480dc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f48080cf880 con 0x7f4808162020 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.634+0000 7f480dc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48080b7b30 con 0x7f48080cf110 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.634+0000 7f480dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48080d2b90 con 0x7f48080d1ac0 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4808162020 0x7f48080cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4808162020 0x7f48080cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48736/0 (socket says 172.21.15.150:48736) 2024-04-15T15:24:25.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 -- 172.21.15.150:0/2712176241 learned_addr learned my addr 172.21.15.150:0/2712176241 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:25.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f4807e00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48080cf110 0x7f4808157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:25.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f47ff400640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080d1ac0 0x7f48080c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:25.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 -- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080d1ac0 msgr2=0x7f48080c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:25.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080d1ac0 0x7f48080c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 -- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48080cf110 msgr2=0x7f4808157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:25.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48080cf110 0x7f4808157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 -- 172.21.15.150:0/2712176241 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48080c9ca0 con 0x7f4808162020 2024-04-15T15:24:25.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.635+0000 7f480d200640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4808162020 0x7f48080cd470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f480000c760 tx=0x7f480000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:25.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.636+0000 7f4804c00640 1 -- 172.21.15.150:0/2712176241 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800017070 con 0x7f4808162020 2024-04-15T15:24:25.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.636+0000 7f4804c00640 1 -- 172.21.15.150:0/2712176241 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480000a640 con 0x7f4808162020 2024-04-15T15:24:25.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.636+0000 7f4804c00640 1 -- 172.21.15.150:0/2712176241 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480000a9f0 con 0x7f4808162020 2024-04-15T15:24:25.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.636+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48080c9f40 con 0x7f4808162020 2024-04-15T15:24:25.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.636+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48081e8470 con 0x7f4808162020 2024-04-15T15:24:25.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.637+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48080755c0 con 0x7f4808162020 2024-04-15T15:24:25.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.638+0000 7f4804c00640 1 -- 172.21.15.150:0/2712176241 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f480001a070 con 0x7f4808162020 2024-04-15T15:24:25.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.639+0000 7f4804c00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47e406c8f0 0x7f47e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:25.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.639+0000 7f4804c00640 1 -- 172.21.15.150:0/2712176241 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f480008b090 con 0x7f4808162020 2024-04-15T15:24:25.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.640+0000 7f4807e00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47e406c8f0 0x7f47e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:25.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.641+0000 7f4807e00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47e406c8f0 0x7f47e406edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009ad0 tx=0x7f47f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:25.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.644+0000 7f4804c00640 1 -- 172.21.15.150:0/2712176241 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48080d0090 con 0x7f4808162020 2024-04-15T15:24:25.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.892+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f48080ceef0 con 0x7f4808162020 2024-04-15T15:24:25.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.892+0000 7f4804c00640 1 -- 172.21.15.150:0/2712176241 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f4800058dd0 con 0x7f4808162020 2024-04-15T15:24:25.894 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47e406c8f0 msgr2=0x7f47e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47e406c8f0 0x7f47e406edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009ad0 tx=0x7f47f8008040 comp rx=0 tx=0).stop 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4808162020 msgr2=0x7f48080cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4808162020 0x7f48080cd470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f480000c760 tx=0x7f480000cc20 comp rx=0 tx=0).stop 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 shutdown_connections 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4808162020 0x7f48080cd470 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47e406c8f0 0x7f47e406edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48080d1ac0 0x7f48080c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 --2- 172.21.15.150:0/2712176241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48080cf110 0x7f4808157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:25.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 >> 172.21.15.150:0/2712176241 conn(0x7f480806cf00 msgr2=0x7f48080b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:25.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 shutdown_connections 2024-04-15T15:24:25.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:25.895+0000 7f480dc00640 1 -- 172.21.15.150:0/2712176241 wait complete. 2024-04-15T15:24:25.917 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-15T15:24:25.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:24:25.917 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-04-15T15:24:25.917 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-15T15:24:25.917 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:24:25.917 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-15T15:24:25.918 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-15T15:24:25.918 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:24:25.918 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:24:25.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:25.918 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-04-15T15:24:25.976 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:24:25.976 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:24:25.976 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:24:25.976 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42597sec preferred_lft 42597sec 2024-04-15T15:24:25.977 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 5a:c2:59:f5:7f:4e brd ff:ff:ff:ff:ff:ff 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::442c:2bff:fe33:cf6c/64 scope link 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 5a:c2:59:f5:7f:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::58c2:59ff:fef5:7f4e/64 scope link 2024-04-15T15:24:25.978 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:25.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:25.979 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:24:26.070 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-15T15:24:26.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:26.075 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-04-15T15:24:26.128 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-04-15T15:24:26.128 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:24:26.128 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:24:26.128 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:24:26.128 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:24:26.129 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:26.129 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:26.129 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:24:26.129 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:24:26.130 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:24:26.130 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:26.232 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:26.356 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:26.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:26.366 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:24:26.416 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:24:26.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:26.417 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-04-15T15:24:26.467 INFO:teuthology.orchestra.run.smithi187.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-15T15:24:26.467 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:26.467 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:26.468 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:24:26.468 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-15T15:24:26.468 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:26.565 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:26.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:26.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:26.663 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-15T15:24:26.752 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:24:26.752 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:24:26.752 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:24:26.752 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:26.752 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-15T15:24:26.752 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:26.753 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:24:26.753 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 66:ed:85:ef:d8:32 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:24:26.753 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:24:26.753 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:26.753 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::64ed:85ff:feef:d832/64 scope link 2024-04-15T15:24:26.753 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:26.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:26.756 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-15T15:24:26.852 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:24:26.852 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:24:26.852 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:24:26.852 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:26.852 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-15T15:24:26.852 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:26.852 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:24:26.853 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 66:ed:85:ef:d8:32 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-15T15:24:26.853 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-15T15:24:26.853 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:26.853 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::64ed:85ff:feef:d832/64 scope link 2024-04-15T15:24:26.853 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:24:26.857 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-15T15:24:26.857 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:26.857 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:26.857 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-15T15:24:26.858 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-15T15:24:26.858 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-15T15:24:26.858 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-15T15:24:26.858 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-15T15:24:26.858 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:26.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:27.176 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:27.185 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:24:27.185 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:24:27.185 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.1 up 2024-04-15T15:24:27.185 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.1 master ceph-brx 2024-04-15T15:24:27.185 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:24:27.290 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:27.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:27.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:24:27.412 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-15T15:24:27.461 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:24:27.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:24:27.462 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:24:27.504 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:24:27.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:27.505 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:24:27.682 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:24:27.683 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:24:27.683 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:24:27.703 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:24:27.703 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:24:27.704 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:24:27.704 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:24:27.704 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:24:27.729 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:24:27.729 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:24:27.729 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:24:27.729 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:24:27.729 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:24:27.729 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:24:27.730 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-04-15T15:24:27.730 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:24:27.731 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-15T15:24:27.731 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:24:27.732 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:24:27.732 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:24:27.732 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:24:27.736 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:24:27.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:27.737 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:24:27.823 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:27.919 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:27.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:27.934 INFO:teuthology.orchestra.run.smithi187.stdout:client4430 2024-04-15T15:24:27.934 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:24:27.934 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:27.978 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:28.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:28.125 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:28.135 INFO:teuthology.orchestra.run.smithi187.stdout:client4430 2024-04-15T15:24:28.135 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:24:28.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:24:28.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:28.251 INFO:teuthology.orchestra.run.smithi187.stdout:client4430 2024-04-15T15:24:28.251 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4.client4430/status') 2024-04-15T15:24:28.354 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:24:28.394 INFO:teuthology.orchestra.run.smithi187.stdout:instance: client.4430 (3)192.168.0.2:0/1979669837 2024-04-15T15:24:28.394 INFO:teuthology.orchestra.run.smithi187.stdout:blocklisted: false 2024-04-15T15:24:28.394 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records in 2024-04-15T15:24:28.394 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records out 2024-04-15T15:24:28.394 INFO:teuthology.orchestra.run.smithi187.stderr:69 bytes copied, 8.9632e-05 s, 770 kB/s 2024-04-15T15:24:28.397 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:24:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:24:28.406 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-15T15:24:28.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:28.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.636+0000 7fa38e200640 1 Processor -- start 2024-04-15T15:24:28.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.636+0000 7fa38e200640 1 -- start start 2024-04-15T15:24:28.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.637+0000 7fa38e200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa3880b3680 0x7fa3880afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:28.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa38e200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa388163060 con 0x7fa388071f50 2024-04-15T15:24:28.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa38e200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3880afae0 con 0x7fa388162c80 2024-04-15T15:24:28.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa38e200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa3880b4ad0 con 0x7fa3880b3680 2024-04-15T15:24:28.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa37fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa3880b3680 0x7fa3880afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:28.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa38d800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa388071f50 0x7fa3880b4f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44800/0 (socket says 172.21.15.150:44800) 2024-04-15T15:24:28.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa38d800640 1 -- 172.21.15.150:0/139067452 learned_addr learned my addr 172.21.15.150:0/139067452 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:28.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa37fe00640 1 -- 172.21.15.150:0/139067452 >> v1:172.21.15.187:6789/0 conn(0x7fa388162c80 legacy=0x7fa3880ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:28.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa37fe00640 1 -- 172.21.15.150:0/139067452 >> v1:172.21.15.150:6789/0 conn(0x7fa388071f50 legacy=0x7fa3880b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:28.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa37fe00640 1 -- 172.21.15.150:0/139067452 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3880b3c60 con 0x7fa3880b3680 2024-04-15T15:24:28.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.638+0000 7fa387e00640 1 -- 172.21.15.150:0/139067452 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508511418 0 0) 0x7fa380001490 con 0x7fa388071f50 2024-04-15T15:24:28.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.639+0000 7fa37fe00640 1 --2- 172.21.15.150:0/139067452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3880b3680 0x7fa3880afd00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa3700097a0 tx=0x7fa37002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a86e50f52359241 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa387e00640 1 -- 172.21.15.150:0/139067452 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370035020 con 0x7fa3880b3680 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa387e00640 1 -- 172.21.15.150:0/139067452 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa370031830 con 0x7fa3880b3680 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa387e00640 1 -- 172.21.15.150:0/139067452 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370031c00 con 0x7fa3880b3680 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa38e200640 1 -- 172.21.15.150:0/139067452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3880b3680 msgr2=0x7fa3880afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa38e200640 1 --2- 172.21.15.150:0/139067452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3880b3680 0x7fa3880afd00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa3700097a0 tx=0x7fa37002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa38e200640 1 -- 172.21.15.150:0/139067452 shutdown_connections 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa38e200640 1 --2- 172.21.15.150:0/139067452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3880b3680 0x7fa3880afd00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa38e200640 1 -- 172.21.15.150:0/139067452 >> 172.21.15.150:0/139067452 conn(0x7fa38806b930 msgr2=0x7fa38806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:28.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.640+0000 7fa38e200640 1 -- 172.21.15.150:0/139067452 shutdown_connections 2024-04-15T15:24:28.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.641+0000 7fa38e200640 1 -- 172.21.15.150:0/139067452 wait complete. 2024-04-15T15:24:28.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.642+0000 7fa38e200640 1 Processor -- start 2024-04-15T15:24:28.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.642+0000 7fa38e200640 1 -- start start 2024-04-15T15:24:28.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.643+0000 7fa38e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa388071f50 0x7fa38814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:28.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.643+0000 7fa38e200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3880b3680 0x7fa3881ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:28.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.643+0000 7fa38e200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa388162c80 0x7fa3881ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:28.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.643+0000 7fa38e200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3880b4500 con 0x7fa388071f50 2024-04-15T15:24:28.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.643+0000 7fa38e200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa3880b4220 con 0x7fa388162c80 2024-04-15T15:24:28.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.643+0000 7fa38e200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa3880b5670 con 0x7fa3880b3680 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.643+0000 7fa38ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa388071f50 0x7fa38814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa38ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa388071f50 0x7fa38814c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35868/0 (socket says 172.21.15.150:35868) 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa38ce00640 1 -- 172.21.15.150:0/2470583255 learned_addr learned my addr 172.21.15.150:0/2470583255 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa37fe00640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3880b3680 0x7fa3881ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa38ce00640 1 -- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3880b3680 msgr2=0x7fa3881ee380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa38ce00640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3880b3680 0x7fa3881ee380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa38ce00640 1 -- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa388162c80 msgr2=0x7fa3881ee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa38ce00640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa388162c80 0x7fa3881ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa38ce00640 1 -- 172.21.15.150:0/2470583255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3881ed2e0 con 0x7fa388071f50 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa37fe00640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3880b3680 0x7fa3881ee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa38ce00640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa388071f50 0x7fa38814c850 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa37800b420 tx=0x7fa37800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.644+0000 7fa385600640 1 -- 172.21.15.150:0/2470583255 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378011830 con 0x7fa388071f50 2024-04-15T15:24:28.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.645+0000 7fa385600640 1 -- 172.21.15.150:0/2470583255 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa378016070 con 0x7fa388071f50 2024-04-15T15:24:28.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.645+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3881ed5c0 con 0x7fa388071f50 2024-04-15T15:24:28.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.645+0000 7fa385600640 1 -- 172.21.15.150:0/2470583255 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378011c10 con 0x7fa388071f50 2024-04-15T15:24:28.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.645+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881edb80 con 0x7fa388071f50 2024-04-15T15:24:28.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.646+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3881edfc0 con 0x7fa388071f50 2024-04-15T15:24:28.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.647+0000 7fa385600640 1 -- 172.21.15.150:0/2470583255 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa37801b460 con 0x7fa388071f50 2024-04-15T15:24:28.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.648+0000 7fa385600640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:28.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.648+0000 7fa385600640 1 -- 172.21.15.150:0/2470583255 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa37808b770 con 0x7fa388071f50 2024-04-15T15:24:28.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.648+0000 7fa37fe00640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:28.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.650+0000 7fa37fe00640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa36406c8f0 0x7fa36406edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa370036000 tx=0x7fa370008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:28.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.652+0000 7fa385600640 1 -- 172.21.15.150:0/2470583255 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa378020020 con 0x7fa388071f50 2024-04-15T15:24:28.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.852+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3880755c0 con 0x7fa388071f50 2024-04-15T15:24:28.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.854+0000 7fa385600640 1 -- 172.21.15.150:0/2470583255 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7fa378059a70 con 0x7fa388071f50 2024-04-15T15:24:28.857 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:28.857 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:28.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.856+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa36406c8f0 msgr2=0x7fa36406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:28.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.856+0000 7fa38e200640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa36406c8f0 0x7fa36406edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa370036000 tx=0x7fa370008030 comp rx=0 tx=0).stop 2024-04-15T15:24:28.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.856+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa388071f50 msgr2=0x7fa38814c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:28.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.856+0000 7fa38e200640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa388071f50 0x7fa38814c850 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa37800b420 tx=0x7fa37800b8e0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.857+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 shutdown_connections 2024-04-15T15:24:28.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.857+0000 7fa38e200640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa388162c80 0x7fa3881ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.857+0000 7fa38e200640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.857+0000 7fa38e200640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3880b3680 0x7fa3881ee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.857+0000 7fa38e200640 1 --2- 172.21.15.150:0/2470583255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa388071f50 0x7fa38814c850 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:28.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.857+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 >> 172.21.15.150:0/2470583255 conn(0x7fa38806b930 msgr2=0x7fa388161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:28.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.857+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 shutdown_connections 2024-04-15T15:24:28.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:28.857+0000 7fa38e200640 1 -- 172.21.15.150:0/2470583255 wait complete. 2024-04-15T15:24:28.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:28.878 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:29.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.086+0000 7fbf33200640 1 Processor -- start 2024-04-15T15:24:29.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.086+0000 7fbf33200640 1 -- start start 2024-04-15T15:24:29.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.088+0000 7fbf33200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf2c15a440 0x7fbf2c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.088+0000 7fbf33200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c06fc80 0x7fbf2c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.088+0000 7fbf33200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2c15d520 con 0x7fbf2c0cd620 2024-04-15T15:24:29.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.088+0000 7fbf33200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c06e280 con 0x7fbf2c15a440 2024-04-15T15:24:29.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.088+0000 7fbf33200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c15d800 con 0x7fbf2c06fc80 2024-04-15T15:24:29.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.088+0000 7fbf2be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf2c15a440 0x7fbf2c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.088+0000 7fbf2be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf2c15a440 0x7fbf2c15cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35872/0 (socket says 172.21.15.150:35872) 2024-04-15T15:24:29.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.089+0000 7fbf2be00640 1 -- 172.21.15.150:0/2876957667 learned_addr learned my addr 172.21.15.150:0/2876957667 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:29.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.089+0000 7fbf23e00640 1 --2- 172.21.15.150:0/2876957667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c06fc80 0x7fbf2c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.089+0000 7fbf2be00640 1 -- 172.21.15.150:0/2876957667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c06fc80 msgr2=0x7fbf2c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.089+0000 7fbf2be00640 1 --2- 172.21.15.150:0/2876957667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c06fc80 0x7fbf2c0cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.089+0000 7fbf2be00640 1 -- 172.21.15.150:0/2876957667 >> v1:172.21.15.187:6789/0 conn(0x7fbf2c0cd620 legacy=0x7fbf2c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.089+0000 7fbf2be00640 1 -- 172.21.15.150:0/2876957667 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c15dae0 con 0x7fbf2c15a440 2024-04-15T15:24:29.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.091+0000 7fbf2be00640 1 --2- 172.21.15.150:0/2876957667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c15a440 0x7fbf2c15cef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009870 tx=0x7fbf1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33a8715421168e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.091+0000 7fbf2b400640 1 -- 172.21.15.150:0/2876957667 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c035020 con 0x7fbf2c15a440 2024-04-15T15:24:29.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.091+0000 7fbf2b400640 1 -- 172.21.15.150:0/2876957667 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c031640 con 0x7fbf2c15a440 2024-04-15T15:24:29.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.091+0000 7fbf2b400640 1 -- 172.21.15.150:0/2876957667 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c02d040 con 0x7fbf2c15a440 2024-04-15T15:24:29.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.092+0000 7fbf33200640 1 -- 172.21.15.150:0/2876957667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c15a440 msgr2=0x7fbf2c15cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.092+0000 7fbf33200640 1 --2- 172.21.15.150:0/2876957667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c15a440 0x7fbf2c15cef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009870 tx=0x7fbf1c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.092+0000 7fbf33200640 1 -- 172.21.15.150:0/2876957667 shutdown_connections 2024-04-15T15:24:29.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.092+0000 7fbf33200640 1 --2- 172.21.15.150:0/2876957667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c06fc80 0x7fbf2c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.092+0000 7fbf33200640 1 --2- 172.21.15.150:0/2876957667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c15a440 0x7fbf2c15cef0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.092+0000 7fbf33200640 1 -- 172.21.15.150:0/2876957667 >> 172.21.15.150:0/2876957667 conn(0x7fbf2c0b2460 msgr2=0x7fbf2c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:29.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.092+0000 7fbf33200640 1 -- 172.21.15.150:0/2876957667 shutdown_connections 2024-04-15T15:24:29.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.092+0000 7fbf33200640 1 -- 172.21.15.150:0/2876957667 wait complete. 2024-04-15T15:24:29.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.094+0000 7fbf33200640 1 Processor -- start 2024-04-15T15:24:29.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.094+0000 7fbf33200640 1 -- start start 2024-04-15T15:24:29.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.095+0000 7fbf33200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c06fc80 0x7fbf2c0af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.095+0000 7fbf33200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf2c0cd620 0x7fbf2c141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.095+0000 7fbf33200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c15a440 0x7fbf2c13ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.095+0000 7fbf33200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c06f510 con 0x7fbf2c06fc80 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.095+0000 7fbf33200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c15ddc0 con 0x7fbf2c0cd620 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.095+0000 7fbf33200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c072860 con 0x7fbf2c15a440 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c06fc80 0x7fbf2c0af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c06fc80 0x7fbf2c0af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35888/0 (socket says 172.21.15.150:35888) 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 -- 172.21.15.150:0/3539461391 learned_addr learned my addr 172.21.15.150:0/3539461391 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 -- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c15a440 msgr2=0x7fbf2c13ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c15a440 0x7fbf2c13ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 -- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf2c0cd620 msgr2=0x7fbf2c141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf2c0cd620 0x7fbf2c141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 -- 172.21.15.150:0/3539461391 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c0ae090 con 0x7fbf2c06fc80 2024-04-15T15:24:29.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.096+0000 7fbf2be00640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c06fc80 0x7fbf2c0af130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c02b930 tx=0x7fbf1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.097+0000 7fbf28c00640 1 -- 172.21.15.150:0/3539461391 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c040070 con 0x7fbf2c06fc80 2024-04-15T15:24:29.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.097+0000 7fbf28c00640 1 -- 172.21.15.150:0/3539461391 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c0399f0 con 0x7fbf2c06fc80 2024-04-15T15:24:29.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.097+0000 7fbf28c00640 1 -- 172.21.15.150:0/3539461391 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c035040 con 0x7fbf2c06fc80 2024-04-15T15:24:29.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.097+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c0ae370 con 0x7fbf2c06fc80 2024-04-15T15:24:29.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.097+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c0ae930 con 0x7fbf2c06fc80 2024-04-15T15:24:29.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.098+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c13d670 con 0x7fbf2c06fc80 2024-04-15T15:24:29.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.099+0000 7fbf28c00640 1 -- 172.21.15.150:0/3539461391 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf1c04b060 con 0x7fbf2c06fc80 2024-04-15T15:24:29.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.100+0000 7fbf28c00640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf0806c8f0 0x7fbf0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.100+0000 7fbf28c00640 1 -- 172.21.15.150:0/3539461391 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbf1c0ad540 con 0x7fbf2c06fc80 2024-04-15T15:24:29.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.101+0000 7fbf23e00640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf0806c8f0 0x7fbf0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.102+0000 7fbf23e00640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf0806c8f0 0x7fbf0806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c1596f0 tx=0x7fbf14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.105+0000 7fbf28c00640 1 -- 172.21.15.150:0/3539461391 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf1c07b8d0 con 0x7fbf2c06fc80 2024-04-15T15:24:29.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.296+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf2c0af670 con 0x7fbf2c06fc80 2024-04-15T15:24:29.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.297+0000 7fbf28c00640 1 -- 172.21.15.150:0/3539461391 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fbf1c07b1d0 con 0x7fbf2c06fc80 2024-04-15T15:24:29.299 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:29.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf0806c8f0 msgr2=0x7fbf0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf0806c8f0 0x7fbf0806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c1596f0 tx=0x7fbf14008040 comp rx=0 tx=0).stop 2024-04-15T15:24:29.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c06fc80 msgr2=0x7fbf2c0af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c06fc80 0x7fbf2c0af130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c02b930 tx=0x7fbf1c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 shutdown_connections 2024-04-15T15:24:29.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf0806c8f0 0x7fbf0806edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf2c15a440 0x7fbf2c13ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf2c0cd620 0x7fbf2c141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 --2- 172.21.15.150:0/3539461391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c06fc80 0x7fbf2c0af130 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 >> 172.21.15.150:0/3539461391 conn(0x7fbf2c0b2460 msgr2=0x7fbf2c15f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:29.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 shutdown_connections 2024-04-15T15:24:29.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.300+0000 7fbf33200640 1 -- 172.21.15.150:0/3539461391 wait complete. 2024-04-15T15:24:29.320 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:29.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.516+0000 7f89b9400640 1 Processor -- start 2024-04-15T15:24:29.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.516+0000 7f89b9400640 1 -- start start 2024-04-15T15:24:29.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.518+0000 7f89b9400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b415c680 0x7f89b4162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.518+0000 7f89b9400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f89b40d2b50 0x7f89b40d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.518+0000 7f89b9400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b40b70d0 con 0x7f89b40b6c70 2024-04-15T15:24:29.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.518+0000 7f89b9400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f89b40d34f0 con 0x7f89b40d2b50 2024-04-15T15:24:29.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.518+0000 7f89b9400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f89b4162dd0 con 0x7f89b415c680 2024-04-15T15:24:29.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.518+0000 7f89abe00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f89b40b6c70 0x7f89b415f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44808/0 (socket says 172.21.15.150:44808) 2024-04-15T15:24:29.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.518+0000 7f89abe00640 1 -- 172.21.15.150:0/535628277 learned_addr learned my addr 172.21.15.150:0/535628277 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3e00640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b415c680 0x7f89b4162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3948247171 0 0) 0x7f89b40b70d0 con 0x7f89b40b6c70 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8998003520 con 0x7f89b40b6c70 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b8a00640 1 --2- 172.21.15.150:0/535628277 >> v2:172.21.15.187:3300/0 conn(0x7f89b40d2b50 0x7f89b40d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3453095309 0 0) 0x7f8998003520 con 0x7f89b40b6c70 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b415c680 msgr2=0x7f89b4162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b415c680 0x7f89b4162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 >> v2:172.21.15.187:3300/0 conn(0x7f89b40d2b50 msgr2=0x7f89b40d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 --2- 172.21.15.150:0/535628277 >> v2:172.21.15.187:3300/0 conn(0x7f89b40d2b50 0x7f89b40d2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b41630b0 con 0x7f89b40b6c70 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f899c001670 con 0x7f89b40b6c70 2024-04-15T15:24:29.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 >> v1:172.21.15.150:6789/0 conn(0x7f89b40b6c70 legacy=0x7f89b415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89980039b0 0x7f8998003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8998004800 0x7f8998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b415c680 0x7f89980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40d34f0 con 0x7f89980039b0 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4162dd0 con 0x7f8998004800 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f899800ba70 con 0x7f89b415c680 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.519+0000 7f89b3e00640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89980039b0 0x7f8998003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.520+0000 7f89abe00640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8998004800 0x7f8998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.520+0000 7f89b3e00640 1 -- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b415c680 msgr2=0x7f89980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.520+0000 7f89b3e00640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b415c680 0x7f89980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.520+0000 7f89b3e00640 1 -- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8998004800 msgr2=0x7f8998004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.520+0000 7f89b3e00640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8998004800 0x7f8998004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.520+0000 7f89b3e00640 1 -- 172.21.15.150:0/535628277 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8998003520 con 0x7f89980039b0 2024-04-15T15:24:29.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.521+0000 7f89b3e00640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89980039b0 0x7f8998003dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89a400ba00 tx=0x7f8998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b152c4f99cbb5fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.521+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4036020 con 0x7f89980039b0 2024-04-15T15:24:29.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.521+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a4032620 con 0x7f89980039b0 2024-04-15T15:24:29.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.521+0000 7f89b3400640 1 -- 172.21.15.150:0/535628277 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a400b600 con 0x7f89980039b0 2024-04-15T15:24:29.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.522+0000 7f89b9400640 1 -- 172.21.15.150:0/535628277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89980039b0 msgr2=0x7f8998003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.522+0000 7f89b9400640 1 --2- 172.21.15.150:0/535628277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89980039b0 0x7f8998003dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89a400ba00 tx=0x7f8998005830 comp rx=0 tx=0).stop 2024-04-15T15:24:29.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.522+0000 7f89b8a00640 1 -- 172.21.15.150:0/535628277 reap_dead start 2024-04-15T15:24:29.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.522+0000 7f89b9400640 1 -- 172.21.15.150:0/535628277 shutdown_connections 2024-04-15T15:24:29.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.522+0000 7f89b9400640 1 -- 172.21.15.150:0/535628277 >> 172.21.15.150:0/535628277 conn(0x7f89b4073480 msgr2=0x7f89b4073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:29.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.522+0000 7f89b9400640 1 -- 172.21.15.150:0/535628277 shutdown_connections 2024-04-15T15:24:29.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.522+0000 7f89b9400640 1 -- 172.21.15.150:0/535628277 wait complete. 2024-04-15T15:24:29.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.523+0000 7f89b9400640 1 Processor -- start 2024-04-15T15:24:29.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.524+0000 7f89b9400640 1 -- start start 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.524+0000 7f89b9400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89980039b0 0x7f89b41f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b9400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b40b81f0 0x7f89b40d4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b9400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89b40b8670 0x7f89b40d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b9400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89b415ca60 con 0x7f89b40b8670 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b9400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89b415f330 con 0x7f89980039b0 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b9400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f89b415ff60 con 0x7f89b40b81f0 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b8a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89b40b8670 0x7f89b40d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b8a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89b40b8670 0x7f89b40d52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35910/0 (socket says 172.21.15.150:35910) 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b8a00640 1 -- 172.21.15.150:0/3199468004 learned_addr learned my addr 172.21.15.150:0/3199468004 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b8a00640 1 -- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b40b81f0 msgr2=0x7f89b40d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89abe00640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b40b81f0 0x7f89b40d4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b3e00640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89980039b0 0x7f89b41f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b8a00640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b40b81f0 0x7f89b40d4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b8a00640 1 -- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89980039b0 msgr2=0x7f89b41f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b8a00640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89980039b0 0x7f89b41f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.525+0000 7f89b8a00640 1 -- 172.21.15.150:0/3199468004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b40b8ef0 con 0x7f89b40b8670 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.526+0000 7f89b8a00640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89b40b8670 0x7f89b40d52a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f89b40b73a0 tx=0x7f89ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.526+0000 7f89b0c00640 1 -- 172.21.15.150:0/3199468004 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac00ed90 con 0x7f89b40b8670 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.526+0000 7f89b0c00640 1 -- 172.21.15.150:0/3199468004 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac019070 con 0x7f89b40b8670 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.526+0000 7f89b0c00640 1 -- 172.21.15.150:0/3199468004 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac014660 con 0x7f89b40b8670 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.526+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b40b91d0 con 0x7f89b40b8670 2024-04-15T15:24:29.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.526+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b40b9790 con 0x7f89b40b8670 2024-04-15T15:24:29.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.527+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b40755c0 con 0x7f89b40b8670 2024-04-15T15:24:29.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.528+0000 7f89b0c00640 1 -- 172.21.15.150:0/3199468004 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89ac014800 con 0x7f89b40b8670 2024-04-15T15:24:29.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.529+0000 7f89b0c00640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f899006c8f0 0x7f899006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.529+0000 7f89b0c00640 1 -- 172.21.15.150:0/3199468004 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f89ac08bf20 con 0x7f89b40b8670 2024-04-15T15:24:29.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.530+0000 7f89b3e00640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f899006c8f0 0x7f899006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.531+0000 7f89b3e00640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f899006c8f0 0x7f899006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f89980057e0 tx=0x7f89a4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.534+0000 7f89b0c00640 1 -- 172.21.15.150:0/3199468004 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89ac05a350 con 0x7f89b40b8670 2024-04-15T15:24:29.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.716+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89b40d0d80 con 0x7f89b40b8670 2024-04-15T15:24:29.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.717+0000 7f89b0c00640 1 -- 172.21.15.150:0/3199468004 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f89ac059c50 con 0x7f89b40b8670 2024-04-15T15:24:29.719 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.719+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f899006c8f0 msgr2=0x7f899006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.719+0000 7f89b9400640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f899006c8f0 0x7f899006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f89980057e0 tx=0x7f89a4030000 comp rx=0 tx=0).stop 2024-04-15T15:24:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.719+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89b40b8670 msgr2=0x7f89b40d52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.719+0000 7f89b9400640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89b40b8670 0x7f89b40d52a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f89b40b73a0 tx=0x7f89ac00e4b0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.719+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 shutdown_connections 2024-04-15T15:24:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.719+0000 7f89b9400640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89b40b8670 0x7f89b40d52a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.719+0000 7f89b9400640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f899006c8f0 0x7f899006edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.719+0000 7f89b9400640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89b40b81f0 0x7f89b40d4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.720+0000 7f89b9400640 1 --2- 172.21.15.150:0/3199468004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89980039b0 0x7f89b41f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.720+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 >> 172.21.15.150:0/3199468004 conn(0x7f89b4073480 msgr2=0x7f89b41614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.720+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 shutdown_connections 2024-04-15T15:24:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.720+0000 7f89b9400640 1 -- 172.21.15.150:0/3199468004 wait complete. 2024-04-15T15:24:29.739 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:29.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.947+0000 7f54fb200640 1 Processor -- start 2024-04-15T15:24:29.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.947+0000 7f54fb200640 1 -- start start 2024-04-15T15:24:29.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.948+0000 7f54fb200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f54f4145170 0x7f54f40cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.949+0000 7f54fb200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f40b4410 con 0x7f54f40ce070 2024-04-15T15:24:29.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.949+0000 7f54fb200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f40b46f0 con 0x7f54f406f260 2024-04-15T15:24:29.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.949+0000 7f54fb200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f54f4145550 con 0x7f54f4145170 2024-04-15T15:24:29.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.949+0000 7f54f3400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f54f40ce070 0x7f54f40b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44814/0 (socket says 172.21.15.150:44814) 2024-04-15T15:24:29.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.949+0000 7f54f3400640 1 -- 172.21.15.150:0/3068384818 learned_addr learned my addr 172.21.15.150:0/3068384818 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.949+0000 7f54f3e00640 1 --2- 172.21.15.150:0/3068384818 >> v2:172.21.15.187:3300/0 conn(0x7f54f4145170 0x7f54f40cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.949+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2651483240 0 0) 0x7f54f40b4410 con 0x7f54f40ce070 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.949+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54d8003520 con 0x7f54f40ce070 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1991346547 0 0) 0x7f54d8003520 con 0x7f54f40ce070 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 >> v2:172.21.15.187:3300/0 conn(0x7f54f4145170 msgr2=0x7f54f40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 --2- 172.21.15.150:0/3068384818 >> v2:172.21.15.187:3300/0 conn(0x7f54f4145170 0x7f54f40cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 >> v1:172.21.15.187:6789/0 conn(0x7f54f406f260 legacy=0x7f54f406f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f40b49d0 con 0x7f54f40ce070 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f54e0001670 con 0x7f54f40ce070 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 >> v1:172.21.15.150:6789/0 conn(0x7f54f40ce070 legacy=0x7f54f40b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8003a00 0x7f54d8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54d8004800 0x7f54d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54d80051c0 0x7f54d800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4145550 con 0x7f54d8003a00 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800e300 con 0x7f54d8004800 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f54d800e500 con 0x7f54d80051c0 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.950+0000 7f54f3e00640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8003a00 0x7f54d8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.951+0000 7f54f3e00640 1 -- 172.21.15.150:0/3068384818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54d80051c0 msgr2=0x7f54d800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.951+0000 7f54f3e00640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54d80051c0 0x7f54d800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.951+0000 7f54f3e00640 1 -- 172.21.15.150:0/3068384818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54d8004800 msgr2=0x7f54d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:29.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.951+0000 7f54f3e00640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54d8004800 0x7f54d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.951+0000 7f54f3e00640 1 -- 172.21.15.150:0/3068384818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d8003520 con 0x7f54d8003a00 2024-04-15T15:24:29.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.951+0000 7f54f9200640 1 -- 172.21.15.150:0/3068384818 reap_dead start 2024-04-15T15:24:29.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.952+0000 7f54f3e00640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8003a00 0x7f54d8003e40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009ac0 tx=0x7f54d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddaee10691e1234 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.952+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4039070 con 0x7f54d8003a00 2024-04-15T15:24:29.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.952+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e402c710 con 0x7f54d8003a00 2024-04-15T15:24:29.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.952+0000 7f54f2a00640 1 -- 172.21.15.150:0/3068384818 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4009690 con 0x7f54d8003a00 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.953+0000 7f54fb200640 1 -- 172.21.15.150:0/3068384818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8003a00 msgr2=0x7f54d8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.953+0000 7f54fb200640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8003a00 0x7f54d8003e40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009ac0 tx=0x7f54d8005830 comp rx=0 tx=0).stop 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.953+0000 7f54fb200640 1 -- 172.21.15.150:0/3068384818 shutdown_connections 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.953+0000 7f54fb200640 1 --2- 172.21.15.150:0/3068384818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8003a00 0x7f54d8003e40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.953+0000 7f54fb200640 1 -- 172.21.15.150:0/3068384818 >> 172.21.15.150:0/3068384818 conn(0x7f54f40b2460 msgr2=0x7f54f40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.953+0000 7f54fb200640 1 -- 172.21.15.150:0/3068384818 shutdown_connections 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.953+0000 7f54fb200640 1 -- 172.21.15.150:0/3068384818 wait complete. 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.954+0000 7f54fb200640 1 Processor -- start 2024-04-15T15:24:29.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.955+0000 7f54fb200640 1 -- start start 2024-04-15T15:24:29.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.955+0000 7f54fb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54d8003a00 0x7f54f40a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.955+0000 7f54fb200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f41f4170 0x7f54f41e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54fb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54f415aae0 0x7f54f415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54fb200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40b4030 con 0x7f54f41f4170 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54fb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40b4f90 con 0x7f54f415aae0 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54fb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f54f40b41b0 con 0x7f54d8003a00 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f41f4170 0x7f54f41e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f41f4170 0x7f54f41e0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35918/0 (socket says 172.21.15.150:35918) 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3400640 1 -- 172.21.15.150:0/4248572669 learned_addr learned my addr 172.21.15.150:0/4248572669 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3400640 1 -- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54d8003a00 msgr2=0x7f54f40a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3e00640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54d8003a00 0x7f54f40a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f9200640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54f415aae0 0x7f54f415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3400640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54d8003a00 0x7f54f40a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3400640 1 -- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54f415aae0 msgr2=0x7f54f415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3400640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54f415aae0 0x7f54f415af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.956+0000 7f54f3400640 1 -- 172.21.15.150:0/4248572669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f40a9da0 con 0x7f54f41f4170 2024-04-15T15:24:29.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.957+0000 7f54f3400640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f41f4170 0x7f54f41e0d20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c700 tx=0x7f54e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.957+0000 7f54ebe00640 1 -- 172.21.15.150:0/4248572669 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0018070 con 0x7f54f41f4170 2024-04-15T15:24:29.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.957+0000 7f54ebe00640 1 -- 172.21.15.150:0/4248572669 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e00072f0 con 0x7f54f41f4170 2024-04-15T15:24:29.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.957+0000 7f54ebe00640 1 -- 172.21.15.150:0/4248572669 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e00047e0 con 0x7f54f41f4170 2024-04-15T15:24:29.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.957+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f40aa080 con 0x7f54f41f4170 2024-04-15T15:24:29.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.957+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f40aa640 con 0x7f54f41f4170 2024-04-15T15:24:29.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.958+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f4068c80 con 0x7f54f41f4170 2024-04-15T15:24:29.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.959+0000 7f54ebe00640 1 -- 172.21.15.150:0/4248572669 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54e0004980 con 0x7f54f41f4170 2024-04-15T15:24:29.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.960+0000 7f54ebe00640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f54d406c8f0 0x7f54d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:29.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.960+0000 7f54ebe00640 1 -- 172.21.15.150:0/4248572669 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f54e008b170 con 0x7f54f41f4170 2024-04-15T15:24:29.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.961+0000 7f54f3e00640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f54d406c8f0 0x7f54d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:29.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.962+0000 7f54f3e00640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f54d406c8f0 0x7f54d406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54e4036000 tx=0x7f54e4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:29.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:29.965+0000 7f54ebe00640 1 -- 172.21.15.150:0/4248572669 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54e0004c70 con 0x7f54f41f4170 2024-04-15T15:24:30.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.164+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54f415b4a0 con 0x7f54f41f4170 2024-04-15T15:24:30.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.165+0000 7f54ebe00640 1 -- 172.21.15.150:0/4248572669 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f54e0059500 con 0x7f54f41f4170 2024-04-15T15:24:30.167 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.167+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f54d406c8f0 msgr2=0x7f54d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.167+0000 7f54fb200640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f54d406c8f0 0x7f54d406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54e4036000 tx=0x7f54e4030000 comp rx=0 tx=0).stop 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.167+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f41f4170 msgr2=0x7f54f41e0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.167+0000 7f54fb200640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f41f4170 0x7f54f41e0d20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c700 tx=0x7f54e000cbc0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.168+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 shutdown_connections 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.168+0000 7f54fb200640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54f415aae0 0x7f54f415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.168+0000 7f54fb200640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54f41f4170 0x7f54f41e0d20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.168+0000 7f54fb200640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f54d406c8f0 0x7f54d406edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.168+0000 7f54fb200640 1 --2- 172.21.15.150:0/4248572669 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54d8003a00 0x7f54f40a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.168+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 >> 172.21.15.150:0/4248572669 conn(0x7f54f40b2460 msgr2=0x7f54f40b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:30.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.168+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 shutdown_connections 2024-04-15T15:24:30.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.168+0000 7f54fb200640 1 -- 172.21.15.150:0/4248572669 wait complete. 2024-04-15T15:24:30.188 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:30.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.387+0000 7fe071000640 1 Processor -- start 2024-04-15T15:24:30.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.387+0000 7fe071000640 1 -- start start 2024-04-15T15:24:30.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe071000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe06c15b860 0x7fe06c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe071000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c06f320 0x7fe06c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe071000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe06c0cdfb0 con 0x7fe06c15b480 2024-04-15T15:24:30.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe071000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe06c15f060 con 0x7fe06c15b860 2024-04-15T15:24:30.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe071000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe06c15fec0 con 0x7fe06c06f320 2024-04-15T15:24:30.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe06b400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe06c15b480 0x7fe06c0cd8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44822/0 (socket says 172.21.15.150:44822) 2024-04-15T15:24:30.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe06b400640 1 -- 172.21.15.150:0/354825640 learned_addr learned my addr 172.21.15.150:0/354825640 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:30.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe06be00640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c06f320 0x7fe06c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785917727 0 0) 0x7fe06c0cdfb0 con 0x7fe06c15b480 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.389+0000 7fe06aa00640 1 --2- 172.21.15.150:0/354825640 >> v2:172.21.15.187:3300/0 conn(0x7fe06c15b860 0x7fe06c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe050003520 con 0x7fe06c15b480 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1808244607 0 0) 0x7fe050003520 con 0x7fe06c15b480 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c06f320 msgr2=0x7fe06c0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c06f320 0x7fe06c0b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 >> v2:172.21.15.187:3300/0 conn(0x7fe06c15b860 msgr2=0x7fe06c06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 --2- 172.21.15.150:0/354825640 >> v2:172.21.15.187:3300/0 conn(0x7fe06c15b860 0x7fe06c06ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c15f2b0 con 0x7fe06c15b480 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fe05c001670 con 0x7fe06c15b480 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 >> v1:172.21.15.150:6789/0 conn(0x7fe06c15b480 legacy=0x7fe06c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c06f320 0x7fe0500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003f30 0x7fe050004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050004d40 0x7fe0500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c15f060 con 0x7fe050004d40 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c15fec0 con 0x7fe050003f30 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe05000dbd0 con 0x7fe06c06f320 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.390+0000 7fe06be00640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050004d40 0x7fe0500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.391+0000 7fe06b400640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c06f320 0x7fe0500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.391+0000 7fe06aa00640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003f30 0x7fe050004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.391+0000 7fe06be00640 1 -- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c06f320 msgr2=0x7fe0500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.391+0000 7fe06be00640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c06f320 0x7fe0500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.391+0000 7fe06be00640 1 -- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003f30 msgr2=0x7fe050004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.391+0000 7fe06be00640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003f30 0x7fe050004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.391+0000 7fe06be00640 1 -- 172.21.15.150:0/354825640 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe050003520 con 0x7fe050004d40 2024-04-15T15:24:30.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.391+0000 7fe06aa00640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050003f30 0x7fe050004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:30.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.392+0000 7fe06be00640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050004d40 0x7fe0500051c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe064009fc0 tx=0x7fe050005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b15e403081a2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:30.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.392+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064039070 con 0x7fe050004d40 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.392+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe06402cc10 con 0x7fe050004d40 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.392+0000 7fe06a000640 1 -- 172.21.15.150:0/354825640 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064009c10 con 0x7fe050004d40 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.393+0000 7fe071000640 1 -- 172.21.15.150:0/354825640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050004d40 msgr2=0x7fe0500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.393+0000 7fe071000640 1 --2- 172.21.15.150:0/354825640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050004d40 0x7fe0500051c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe064009fc0 tx=0x7fe050005830 comp rx=0 tx=0).stop 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.393+0000 7fe06be00640 1 -- 172.21.15.150:0/354825640 reap_dead start 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.393+0000 7fe071000640 1 -- 172.21.15.150:0/354825640 shutdown_connections 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.393+0000 7fe071000640 1 -- 172.21.15.150:0/354825640 >> 172.21.15.150:0/354825640 conn(0x7fe06c06b930 msgr2=0x7fe06c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.393+0000 7fe071000640 1 -- 172.21.15.150:0/354825640 shutdown_connections 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.393+0000 7fe071000640 1 -- 172.21.15.150:0/354825640 wait complete. 2024-04-15T15:24:30.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.395+0000 7fe071000640 1 Processor -- start 2024-04-15T15:24:30.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.395+0000 7fe071000640 1 -- start start 2024-04-15T15:24:30.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe071000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050004d40 0x7fe06c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe071000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe06c1397d0 0x7fe06c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe071000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c158a70 0x7fe06c14b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe071000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c15fb50 con 0x7fe06c1397d0 2024-04-15T15:24:30.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe071000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c15f870 con 0x7fe050004d40 2024-04-15T15:24:30.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe071000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe06c0cd550 con 0x7fe06c158a70 2024-04-15T15:24:30.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe06c1397d0 0x7fe06c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe06c1397d0 0x7fe06c14e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35934/0 (socket says 172.21.15.150:35934) 2024-04-15T15:24:30.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06aa00640 1 -- 172.21.15.150:0/1125385004 learned_addr learned my addr 172.21.15.150:0/1125385004 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:30.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06b400640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050004d40 0x7fe06c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06aa00640 1 -- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c158a70 msgr2=0x7fe06c14b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06be00640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c158a70 0x7fe06c14b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06aa00640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c158a70 0x7fe06c14b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06aa00640 1 -- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050004d40 msgr2=0x7fe06c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06aa00640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050004d40 0x7fe06c15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06aa00640 1 -- 172.21.15.150:0/1125385004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c159010 con 0x7fe06c1397d0 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.396+0000 7fe06b400640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050004d40 0x7fe06c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.397+0000 7fe06aa00640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe06c1397d0 0x7fe06c14e580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe058010a00 tx=0x7fe058010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.397+0000 7fe063400640 1 -- 172.21.15.150:0/1125385004 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05800f040 con 0x7fe06c1397d0 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.397+0000 7fe063400640 1 -- 172.21.15.150:0/1125385004 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe05800ebc0 con 0x7fe06c1397d0 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.397+0000 7fe063400640 1 -- 172.21.15.150:0/1125385004 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05801c400 con 0x7fe06c1397d0 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.397+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06c1592f0 con 0x7fe06c1397d0 2024-04-15T15:24:30.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.397+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06c1598b0 con 0x7fe06c1397d0 2024-04-15T15:24:30.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.398+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe06c0755c0 con 0x7fe06c1397d0 2024-04-15T15:24:30.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.399+0000 7fe063400640 1 -- 172.21.15.150:0/1125385004 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe05801c5a0 con 0x7fe06c1397d0 2024-04-15T15:24:30.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.400+0000 7fe063400640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe04c06c8c0 0x7fe04c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.400+0000 7fe063400640 1 -- 172.21.15.150:0/1125385004 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe05808c250 con 0x7fe06c1397d0 2024-04-15T15:24:30.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.401+0000 7fe06b400640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe04c06c8c0 0x7fe04c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.402+0000 7fe06b400640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe04c06c8c0 0x7fe04c06ed80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0017e0 tx=0x7fe05c000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:30.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.405+0000 7fe063400640 1 -- 172.21.15.150:0/1125385004 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe058018030 con 0x7fe06c1397d0 2024-04-15T15:24:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.608+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe06c0b3930 con 0x7fe06c1397d0 2024-04-15T15:24:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.610+0000 7fe063400640 1 -- 172.21.15.150:0/1125385004 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7fe05805a020 con 0x7fe06c1397d0 2024-04-15T15:24:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:30.612 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe04c06c8c0 msgr2=0x7fe04c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe04c06c8c0 0x7fe04c06ed80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0017e0 tx=0x7fe05c000ed0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe06c1397d0 msgr2=0x7fe06c14e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe06c1397d0 0x7fe06c14e580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe058010a00 tx=0x7fe058010ec0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 shutdown_connections 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe04c06c8c0 0x7fe04c06ed80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe06c158a70 0x7fe06c14b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe06c1397d0 0x7fe06c14e580 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 --2- 172.21.15.150:0/1125385004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe050004d40 0x7fe06c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 >> 172.21.15.150:0/1125385004 conn(0x7fe06c06b930 msgr2=0x7fe06c155250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 shutdown_connections 2024-04-15T15:24:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.613+0000 7fe071000640 1 -- 172.21.15.150:0/1125385004 wait complete. 2024-04-15T15:24:30.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:30.634 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.834+0000 7f1ee5200640 1 Processor -- start 2024-04-15T15:24:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.834+0000 7f1ee5200640 1 -- start start 2024-04-15T15:24:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.836+0000 7f1ee5200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ee006fc80 0x7f1ee015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.836+0000 7f1ee5200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee0072150 0x7f1ee00725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.836+0000 7f1ee5200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ee015d520 con 0x7f1ee015a440 2024-04-15T15:24:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.836+0000 7f1ee5200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1ee006e280 con 0x7f1ee006fc80 2024-04-15T15:24:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.836+0000 7f1ee5200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee015d800 con 0x7f1ee0072150 2024-04-15T15:24:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.836+0000 7f1edde00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1ee015a440 0x7f1ee00cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44834/0 (socket says 172.21.15.150:44834) 2024-04-15T15:24:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.836+0000 7f1edde00640 1 -- 172.21.15.150:0/3557908536 learned_addr learned my addr 172.21.15.150:0/3557908536 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.836+0000 7f1ede800640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee0072150 0x7f1ee00725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edd400640 1 --2- 172.21.15.150:0/3557908536 >> v2:172.21.15.187:3300/0 conn(0x7f1ee006fc80 0x7f1ee015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733057816 0 0) 0x7f1ee015d520 con 0x7f1ee015a440 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ec4003520 con 0x7f1ee015a440 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 355986711 0 0) 0x7f1ec4003520 con 0x7f1ee015a440 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee0072150 msgr2=0x7f1ee00725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee0072150 0x7f1ee00725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 >> v2:172.21.15.187:3300/0 conn(0x7f1ee006fc80 msgr2=0x7f1ee015cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 --2- 172.21.15.150:0/3557908536 >> v2:172.21.15.187:3300/0 conn(0x7f1ee006fc80 0x7f1ee015cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee015dae0 con 0x7f1ee015a440 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f1ed0001690 con 0x7f1ee015a440 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edd400640 1 --2- 172.21.15.150:0/3557908536 >> v2:172.21.15.187:3300/0 conn(0x7f1ee006fc80 0x7f1ee015cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 >> v1:172.21.15.150:6789/0 conn(0x7f1ee015a440 legacy=0x7f1ee00cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec40039b0 0x7f1ec4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec4004800 0x7f1ec4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.837+0000 7f1edca00640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee0072150 0x7f1ec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee006e280 con 0x7f1ec4004800 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee015d800 con 0x7f1ec40039b0 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec400b9c0 con 0x7f1ee0072150 2024-04-15T15:24:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edd400640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec4004800 0x7f1ec4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1ede800640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee0072150 0x7f1ec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edde00640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec40039b0 0x7f1ec4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edd400640 1 -- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee0072150 msgr2=0x7f1ec40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:30.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edd400640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee0072150 0x7f1ec40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edd400640 1 -- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec40039b0 msgr2=0x7f1ec4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:30.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edd400640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec40039b0 0x7f1ec4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.838+0000 7f1edd400640 1 -- 172.21.15.150:0/3557908536 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec4003520 con 0x7f1ec4004800 2024-04-15T15:24:30.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.840+0000 7f1edd400640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec4004800 0x7f1ec4004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009c60 tx=0x7f1ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba80e8e16071822e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:30.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.840+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc009860 con 0x7f1ec4004800 2024-04-15T15:24:30.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.840+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ecc004020 con 0x7f1ec4004800 2024-04-15T15:24:30.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.840+0000 7f1edca00640 1 -- 172.21.15.150:0/3557908536 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc004470 con 0x7f1ec4004800 2024-04-15T15:24:30.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.841+0000 7f1ee5200640 1 -- 172.21.15.150:0/3557908536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec4004800 msgr2=0x7f1ec4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.841+0000 7f1ee5200640 1 --2- 172.21.15.150:0/3557908536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec4004800 0x7f1ec4004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009c60 tx=0x7f1ec4005830 comp rx=0 tx=0).stop 2024-04-15T15:24:30.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.841+0000 7f1ede800640 1 -- 172.21.15.150:0/3557908536 reap_dead start 2024-04-15T15:24:30.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.841+0000 7f1ee5200640 1 -- 172.21.15.150:0/3557908536 shutdown_connections 2024-04-15T15:24:30.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.841+0000 7f1ee5200640 1 -- 172.21.15.150:0/3557908536 >> 172.21.15.150:0/3557908536 conn(0x7f1ee00b2460 msgr2=0x7f1ee00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:30.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.841+0000 7f1ee5200640 1 -- 172.21.15.150:0/3557908536 shutdown_connections 2024-04-15T15:24:30.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.841+0000 7f1ee5200640 1 -- 172.21.15.150:0/3557908536 wait complete. 2024-04-15T15:24:30.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.843+0000 7f1ee5200640 1 Processor -- start 2024-04-15T15:24:30.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.843+0000 7f1ee5200640 1 -- start start 2024-04-15T15:24:30.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.844+0000 7f1ee5200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec4004800 0x7f1ee01435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.844+0000 7f1ee5200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee0140090 0x7f1ee00ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.844+0000 7f1ee5200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee00ada70 0x7f1ee0141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.844+0000 7f1ee5200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee015ddc0 con 0x7f1ee0140090 2024-04-15T15:24:30.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.844+0000 7f1ee5200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee015a820 con 0x7f1ec4004800 2024-04-15T15:24:30.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.844+0000 7f1ee5200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee006f510 con 0x7f1ee00ada70 2024-04-15T15:24:30.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.844+0000 7f1edd400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee0140090 0x7f1ee00ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edd400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee0140090 0x7f1ee00ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35954/0 (socket says 172.21.15.150:35954) 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edd400640 1 -- 172.21.15.150:0/276350558 learned_addr learned my addr 172.21.15.150:0/276350558 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edde00640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec4004800 0x7f1ee01435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edd400640 1 -- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee00ada70 msgr2=0x7f1ee0141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edd400640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee00ada70 0x7f1ee0141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edd400640 1 -- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec4004800 msgr2=0x7f1ee01435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edd400640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec4004800 0x7f1ee01435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edd400640 1 -- 172.21.15.150:0/276350558 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee0143b00 con 0x7f1ee0140090 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.845+0000 7f1edd400640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee0140090 0x7f1ee00ad530 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009c60 tx=0x7f1ecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.846+0000 7f1ed6000640 1 -- 172.21.15.150:0/276350558 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc043020 con 0x7f1ee0140090 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.846+0000 7f1ed6000640 1 -- 172.21.15.150:0/276350558 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ecc02c950 con 0x7f1ee0140090 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.846+0000 7f1ed6000640 1 -- 172.21.15.150:0/276350558 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc009860 con 0x7f1ee0140090 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.846+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee0140b40 con 0x7f1ee0140090 2024-04-15T15:24:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.846+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee0141100 con 0x7f1ee0140090 2024-04-15T15:24:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.847+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee0068c80 con 0x7f1ee0140090 2024-04-15T15:24:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.848+0000 7f1ed6000640 1 -- 172.21.15.150:0/276350558 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ecc046020 con 0x7f1ee0140090 2024-04-15T15:24:30.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.849+0000 7f1ed6000640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ec006c920 0x7f1ec006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:30.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.849+0000 7f1ed6000640 1 -- 172.21.15.150:0/276350558 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1ecc0ada00 con 0x7f1ee0140090 2024-04-15T15:24:30.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.850+0000 7f1edde00640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ec006c920 0x7f1ec006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.851+0000 7f1edde00640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ec006c920 0x7f1ec006ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000f60 tx=0x7f1ed0001cb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:30.854+0000 7f1ed6000640 1 -- 172.21.15.150:0/276350558 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ecc04d030 con 0x7f1ee0140090 2024-04-15T15:24:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.053+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ee00b2010 con 0x7f1ee0140090 2024-04-15T15:24:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.054+0000 7f1ed6000640 1 -- 172.21.15.150:0/276350558 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f1ee0143b00 con 0x7f1ee0140090 2024-04-15T15:24:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ec006c920 msgr2=0x7f1ec006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ec006c920 0x7f1ec006ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000f60 tx=0x7f1ed0001cb0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee0140090 msgr2=0x7f1ee00ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee0140090 0x7f1ee00ad530 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009c60 tx=0x7f1ecc004730 comp rx=0 tx=0).stop 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 shutdown_connections 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1ec006c920 0x7f1ec006ede0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee00ada70 0x7f1ee0141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee0140090 0x7f1ee00ad530 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 --2- 172.21.15.150:0/276350558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec4004800 0x7f1ee01435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 >> 172.21.15.150:0/276350558 conn(0x7f1ee00b2460 msgr2=0x7f1ee00cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 shutdown_connections 2024-04-15T15:24:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.057+0000 7f1ee5200640 1 -- 172.21.15.150:0/276350558 wait complete. 2024-04-15T15:24:31.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:31.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.290+0000 7fb0d7000640 1 Processor -- start 2024-04-15T15:24:31.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.290+0000 7fb0d7000640 1 -- start start 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.291+0000 7fb0d7000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0d0145170 0x7fb0d00cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.291+0000 7fb0d7000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb0d00ce070 0x7fb0d00b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.291+0000 7fb0d7000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d00b4410 con 0x7fb0d00b59f0 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.291+0000 7fb0d7000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d0145550 con 0x7fb0d0145170 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.291+0000 7fb0d7000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d00b46f0 con 0x7fb0d00ce070 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0cfe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0d0145170 0x7fb0d00cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0cfe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0d0145170 0x7fb0d00cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35964/0 (socket says 172.21.15.150:35964) 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0cfe00640 1 -- 172.21.15.150:0/702491224 learned_addr learned my addr 172.21.15.150:0/702491224 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0d5000640 1 --1- 172.21.15.150:0/702491224 >> v1:172.21.15.150:6789/0 conn(0x7fb0d00b59f0 0x7fb0d006f260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44844/0 (socket says 172.21.15.150:44844) 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0c7400640 1 --2- 172.21.15.150:0/702491224 >> v2:172.21.15.187:3300/0 conn(0x7fb0d00ce070 0x7fb0d00b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0cfe00640 1 -- 172.21.15.150:0/702491224 >> v2:172.21.15.187:3300/0 conn(0x7fb0d00ce070 msgr2=0x7fb0d00b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0cfe00640 1 --2- 172.21.15.150:0/702491224 >> v2:172.21.15.187:3300/0 conn(0x7fb0d00ce070 0x7fb0d00b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0cfe00640 1 -- 172.21.15.150:0/702491224 >> v1:172.21.15.150:6789/0 conn(0x7fb0d00b59f0 legacy=0x7fb0d006f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.292+0000 7fb0cfe00640 1 -- 172.21.15.150:0/702491224 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d00b49d0 con 0x7fb0d0145170 2024-04-15T15:24:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.293+0000 7fb0cfe00640 1 --2- 172.21.15.150:0/702491224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d00cdb30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb0d01456d0 tx=0x7fb0c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8520785c1cd428 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0cf400640 1 -- 172.21.15.150:0/702491224 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0034070 con 0x7fb0d0145170 2024-04-15T15:24:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0cf400640 1 -- 172.21.15.150:0/702491224 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c0030860 con 0x7fb0d0145170 2024-04-15T15:24:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0cf400640 1 -- 172.21.15.150:0/702491224 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0030c30 con 0x7fb0d0145170 2024-04-15T15:24:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0d7000640 1 -- 172.21.15.150:0/702491224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 msgr2=0x7fb0d00cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0d7000640 1 --2- 172.21.15.150:0/702491224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d00cdb30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb0d01456d0 tx=0x7fb0c002b9a0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0d7000640 1 -- 172.21.15.150:0/702491224 shutdown_connections 2024-04-15T15:24:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0d7000640 1 --2- 172.21.15.150:0/702491224 >> v2:172.21.15.187:3300/0 conn(0x7fb0d00ce070 0x7fb0d00b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0d7000640 1 --2- 172.21.15.150:0/702491224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d00cdb30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.294+0000 7fb0d7000640 1 -- 172.21.15.150:0/702491224 >> 172.21.15.150:0/702491224 conn(0x7fb0d00b2460 msgr2=0x7fb0d00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.295+0000 7fb0d7000640 1 -- 172.21.15.150:0/702491224 shutdown_connections 2024-04-15T15:24:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.295+0000 7fb0d7000640 1 -- 172.21.15.150:0/702491224 wait complete. 2024-04-15T15:24:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.297+0000 7fb0d7000640 1 Processor -- start 2024-04-15T15:24:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.297+0000 7fb0d7000640 1 -- start start 2024-04-15T15:24:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.298+0000 7fb0d7000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d00b59f0 0x7fb0d013f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.298+0000 7fb0d7000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0d00ce070 0x7fb0d01413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.298+0000 7fb0d7000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d0143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.298+0000 7fb0d7000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d00b4030 con 0x7fb0d0145170 2024-04-15T15:24:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.298+0000 7fb0d7000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d00b4f90 con 0x7fb0d00b59f0 2024-04-15T15:24:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.298+0000 7fb0d7000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d00b41b0 con 0x7fb0d00ce070 2024-04-15T15:24:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.298+0000 7fb0d5000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d0143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0d5000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d0143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35978/0 (socket says 172.21.15.150:35978) 2024-04-15T15:24:31.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0d5000640 1 -- 172.21.15.150:0/725945506 learned_addr learned my addr 172.21.15.150:0/725945506 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0cfe00640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d00b59f0 0x7fb0d013f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0c7400640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0d00ce070 0x7fb0d01413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0d5000640 1 -- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0d00ce070 msgr2=0x7fb0d01413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0d5000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0d00ce070 0x7fb0d01413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0d5000640 1 -- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d00b59f0 msgr2=0x7fb0d013f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0d5000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d00b59f0 0x7fb0d013f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0d5000640 1 -- 172.21.15.150:0/725945506 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d01e1c30 con 0x7fb0d0145170 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.299+0000 7fb0d5000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d0143240 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800c780 tx=0x7fb0c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.300+0000 7fb0ccc00640 1 -- 172.21.15.150:0/725945506 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8017070 con 0x7fb0d0145170 2024-04-15T15:24:31.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.300+0000 7fb0ccc00640 1 -- 172.21.15.150:0/725945506 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c800a640 con 0x7fb0d0145170 2024-04-15T15:24:31.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.300+0000 7fb0ccc00640 1 -- 172.21.15.150:0/725945506 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800aa30 con 0x7fb0d0145170 2024-04-15T15:24:31.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.300+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d01e4900 con 0x7fb0d0145170 2024-04-15T15:24:31.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.300+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d01e4ec0 con 0x7fb0d0145170 2024-04-15T15:24:31.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.301+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d0068c80 con 0x7fb0d0145170 2024-04-15T15:24:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.302+0000 7fb0ccc00640 1 -- 172.21.15.150:0/725945506 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0c801a070 con 0x7fb0d0145170 2024-04-15T15:24:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.304+0000 7fb0ccc00640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.304+0000 7fb0ccc00640 1 -- 172.21.15.150:0/725945506 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb0c808b0d0 con 0x7fb0d0145170 2024-04-15T15:24:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.304+0000 7fb0cfe00640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.305+0000 7fb0cfe00640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0036000 tx=0x7fb0c003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:31.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.308+0000 7fb0ccc00640 1 -- 172.21.15.150:0/725945506 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0c8058ea0 con 0x7fb0d0145170 2024-04-15T15:24:31.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.513+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb0d01e5090 con 0x7fb0d0145170 2024-04-15T15:24:31.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.514+0000 7fb0ccc00640 1 -- 172.21.15.150:0/725945506 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fb0c805d1a0 con 0x7fb0d0145170 2024-04-15T15:24:31.515 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:31.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb0ac06c8f0 msgr2=0x7fb0ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0036000 tx=0x7fb0c003b040 comp rx=0 tx=0).stop 2024-04-15T15:24:31.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 msgr2=0x7fb0d0143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d0143240 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800c780 tx=0x7fb0c800cc40 comp rx=0 tx=0).stop 2024-04-15T15:24:31.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 shutdown_connections 2024-04-15T15:24:31.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0d0145170 0x7fb0d0143240 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0d00ce070 0x7fb0d01413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 --2- 172.21.15.150:0/725945506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d00b59f0 0x7fb0d013f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.516+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 >> 172.21.15.150:0/725945506 conn(0x7fb0d00b2460 msgr2=0x7fb0d0072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:31.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.517+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 shutdown_connections 2024-04-15T15:24:31.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.517+0000 7fb0d7000640 1 -- 172.21.15.150:0/725945506 wait complete. 2024-04-15T15:24:31.538 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:31.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.754+0000 7f29faa00640 1 Processor -- start 2024-04-15T15:24:31.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.754+0000 7f29faa00640 1 -- start start 2024-04-15T15:24:31.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.756+0000 7f29faa00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f29f415a440 0x7f29f415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.756+0000 7f29faa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29f4072150 0x7f29f4072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.756+0000 7f29faa00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f415d520 con 0x7f29f406fc80 2024-04-15T15:24:31.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.756+0000 7f29faa00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f29f406e280 con 0x7f29f415a440 2024-04-15T15:24:31.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.756+0000 7f29faa00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f29f4072ad0 con 0x7f29f4072150 2024-04-15T15:24:31.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.756+0000 7f29f3400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f29f406fc80 0x7f29f40cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44848/0 (socket says 172.21.15.150:44848) 2024-04-15T15:24:31.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.756+0000 7f29f3e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f29f415a440 0x7f29f415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.756+0000 7f29f3400640 1 -- 172.21.15.150:0/449389206 learned_addr learned my addr 172.21.15.150:0/449389206 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:31.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.757+0000 7f29f8a00640 1 --2- 172.21.15.150:0/449389206 >> v2:172.21.15.187:3300/0 conn(0x7f29f4072150 0x7f29f4072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.757+0000 7f29f2a00640 1 -- 172.21.15.150:0/449389206 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1088218236 0 0) 0x7f29f415d520 con 0x7f29f406fc80 2024-04-15T15:24:31.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.757+0000 7f29f2a00640 1 -- 172.21.15.150:0/449389206 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29d4003520 con 0x7f29f406fc80 2024-04-15T15:24:31.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.757+0000 7f29f3e00640 1 -- 172.21.15.150:0/449389206 >> v2:172.21.15.187:3300/0 conn(0x7f29f4072150 msgr2=0x7f29f4072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:31.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.757+0000 7f29f3e00640 1 --2- 172.21.15.150:0/449389206 >> v2:172.21.15.187:3300/0 conn(0x7f29f4072150 0x7f29f4072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.757+0000 7f29f3e00640 1 -- 172.21.15.150:0/449389206 >> v1:172.21.15.150:6789/0 conn(0x7f29f406fc80 legacy=0x7f29f40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.757+0000 7f29f3e00640 1 -- 172.21.15.150:0/449389206 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f415d800 con 0x7f29f415a440 2024-04-15T15:24:31.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.758+0000 7f29f3e00640 1 --2- 172.21.15.150:0/449389206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f415a440 0x7f29f415cef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f29f406fba0 tx=0x7f29e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afa2ae73046f5df server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:31.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.759+0000 7f29f2a00640 1 -- 172.21.15.150:0/449389206 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4035020 con 0x7f29f415a440 2024-04-15T15:24:31.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.759+0000 7f29f2a00640 1 -- 172.21.15.150:0/449389206 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e4031830 con 0x7f29f415a440 2024-04-15T15:24:31.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.759+0000 7f29f2a00640 1 -- 172.21.15.150:0/449389206 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4031be0 con 0x7f29f415a440 2024-04-15T15:24:31.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.759+0000 7f29faa00640 1 -- 172.21.15.150:0/449389206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f415a440 msgr2=0x7f29f415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.759+0000 7f29faa00640 1 --2- 172.21.15.150:0/449389206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f415a440 0x7f29f415cef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f29f406fba0 tx=0x7f29e402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.759+0000 7f29faa00640 1 -- 172.21.15.150:0/449389206 shutdown_connections 2024-04-15T15:24:31.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.759+0000 7f29faa00640 1 --2- 172.21.15.150:0/449389206 >> v2:172.21.15.187:3300/0 conn(0x7f29f4072150 0x7f29f4072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.759+0000 7f29faa00640 1 --2- 172.21.15.150:0/449389206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f415a440 0x7f29f415cef0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.760+0000 7f29faa00640 1 -- 172.21.15.150:0/449389206 >> 172.21.15.150:0/449389206 conn(0x7f29f40b2460 msgr2=0x7f29f40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:31.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.760+0000 7f29faa00640 1 -- 172.21.15.150:0/449389206 shutdown_connections 2024-04-15T15:24:31.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.760+0000 7f29faa00640 1 -- 172.21.15.150:0/449389206 wait complete. 2024-04-15T15:24:31.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.761+0000 7f29faa00640 1 Processor -- start 2024-04-15T15:24:31.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.761+0000 7f29faa00640 1 -- start start 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.762+0000 7f29faa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f406fc80 0x7f29f415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.762+0000 7f29faa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f4072150 0x7f29f41f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.762+0000 7f29faa00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f415a440 0x7f29f41420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.762+0000 7f29faa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f29f415a820 con 0x7f29f406fc80 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.762+0000 7f29faa00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29f40cd7f0 con 0x7f29f415a440 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.762+0000 7f29faa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29f415ddc0 con 0x7f29f4072150 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f406fc80 0x7f29f415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f406fc80 0x7f29f415ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35988/0 (socket says 172.21.15.150:35988) 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 -- 172.21.15.150:0/2699318713 learned_addr learned my addr 172.21.15.150:0/2699318713 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 -- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f4072150 msgr2=0x7f29f41f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f4072150 0x7f29f41f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 -- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f415a440 msgr2=0x7f29f41420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f415a440 0x7f29f41420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 -- 172.21.15.150:0/2699318713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f4144bf0 con 0x7f29f406fc80 2024-04-15T15:24:31.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.763+0000 7f29f3e00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f406fc80 0x7f29f415ca10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f29e402b910 tx=0x7f29e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:31.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.764+0000 7f29ebe00640 1 -- 172.21.15.150:0/2699318713 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4035050 con 0x7f29f406fc80 2024-04-15T15:24:31.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.764+0000 7f29ebe00640 1 -- 172.21.15.150:0/2699318713 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e4003610 con 0x7f29f406fc80 2024-04-15T15:24:31.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.764+0000 7f29ebe00640 1 -- 172.21.15.150:0/2699318713 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e40039e0 con 0x7f29f406fc80 2024-04-15T15:24:31.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.764+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f4144630 con 0x7f29f406fc80 2024-04-15T15:24:31.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.764+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f40c5770 con 0x7f29f406fc80 2024-04-15T15:24:31.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.765+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f4068c80 con 0x7f29f406fc80 2024-04-15T15:24:31.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.766+0000 7f29ebe00640 1 -- 172.21.15.150:0/2699318713 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29e4038020 con 0x7f29f406fc80 2024-04-15T15:24:31.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.767+0000 7f29ebe00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d006c920 0x7f29d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:31.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.767+0000 7f29ebe00640 1 -- 172.21.15.150:0/2699318713 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f29e40ad940 con 0x7f29f406fc80 2024-04-15T15:24:31.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.767+0000 7f29f3400640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d006c920 0x7f29d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:31.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.768+0000 7f29f3400640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d006c920 0x7f29d006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29dc000f60 tx=0x7f29dc0078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:31.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.771+0000 7f29ebe00640 1 -- 172.21.15.150:0/2699318713 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29e407bcd0 con 0x7f29f406fc80 2024-04-15T15:24:31.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.971+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f29f40b2a50 con 0x7f29f406fc80 2024-04-15T15:24:31.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.972+0000 7f29ebe00640 1 -- 172.21.15.150:0/2699318713 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f29e407b5d0 con 0x7f29f406fc80 2024-04-15T15:24:31.974 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:31.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.974+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d006c920 msgr2=0x7f29d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.974+0000 7f29faa00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d006c920 0x7f29d006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29dc000f60 tx=0x7f29dc0078f0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.974+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f406fc80 msgr2=0x7f29f415ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.974+0000 7f29faa00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f406fc80 0x7f29f415ca10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f29e402b910 tx=0x7f29e40046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.974+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 shutdown_connections 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.974+0000 7f29faa00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f415a440 0x7f29f41420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.974+0000 7f29faa00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f29d006c920 0x7f29d006ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.975+0000 7f29faa00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f4072150 0x7f29f41f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.975+0000 7f29faa00640 1 --2- 172.21.15.150:0/2699318713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29f406fc80 0x7f29f415ca10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.975+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 >> 172.21.15.150:0/2699318713 conn(0x7f29f40b2460 msgr2=0x7f29f40adcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.975+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 shutdown_connections 2024-04-15T15:24:31.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:31.975+0000 7f29faa00640 1 -- 172.21.15.150:0/2699318713 wait complete. 2024-04-15T15:24:31.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:32.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.197+0000 7fa3c3200640 1 Processor -- start 2024-04-15T15:24:32.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.197+0000 7fa3c3200640 1 -- start start 2024-04-15T15:24:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c40cee00 0x7fa3c406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c3200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa3c4071f50 0x7fa3c40723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c3200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c415dbb0 con 0x7fa3c415cbe0 2024-04-15T15:24:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c3200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c406f4f0 con 0x7fa3c4071f50 2024-04-15T15:24:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c40728f0 con 0x7fa3c40cee00 2024-04-15T15:24:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c1400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa3c4071f50 0x7fa3c40723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c1400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa3c4071f50 0x7fa3c40723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35996/0 (socket says 172.21.15.150:35996) 2024-04-15T15:24:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c1400640 1 -- 172.21.15.150:0/3665642016 learned_addr learned my addr 172.21.15.150:0/3665642016 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:32.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.199+0000 7fa3c1e00640 1 --2- 172.21.15.150:0/3665642016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c40cee00 0x7fa3c406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.200+0000 7fa3c1400640 1 -- 172.21.15.150:0/3665642016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c40cee00 msgr2=0x7fa3c406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:32.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.200+0000 7fa3c1400640 1 --2- 172.21.15.150:0/3665642016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c40cee00 0x7fa3c406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.200+0000 7fa3c1400640 1 -- 172.21.15.150:0/3665642016 >> v1:172.21.15.187:6789/0 conn(0x7fa3c415cbe0 legacy=0x7fa3c415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:32.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.200+0000 7fa3c1400640 1 -- 172.21.15.150:0/3665642016 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c415d8d0 con 0x7fa3c4071f50 2024-04-15T15:24:32.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.201+0000 7fa3c1400640 1 --2- 172.21.15.150:0/3665642016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c4071f50 0x7fa3c40723b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0098a0 tx=0x7fa3ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae0995ad36966b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:32.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c0a00640 1 -- 172.21.15.150:0/3665642016 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac035020 con 0x7fa3c4071f50 2024-04-15T15:24:32.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c0a00640 1 -- 172.21.15.150:0/3665642016 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ac031640 con 0x7fa3c4071f50 2024-04-15T15:24:32.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c0a00640 1 -- 172.21.15.150:0/3665642016 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac02d040 con 0x7fa3c4071f50 2024-04-15T15:24:32.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c3200640 1 -- 172.21.15.150:0/3665642016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c4071f50 msgr2=0x7fa3c40723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c3200640 1 --2- 172.21.15.150:0/3665642016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c4071f50 0x7fa3c40723b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0098a0 tx=0x7fa3ac02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c3200640 1 -- 172.21.15.150:0/3665642016 shutdown_connections 2024-04-15T15:24:32.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c3200640 1 --2- 172.21.15.150:0/3665642016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c4071f50 0x7fa3c40723b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c3200640 1 --2- 172.21.15.150:0/3665642016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c40cee00 0x7fa3c406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.202+0000 7fa3c3200640 1 -- 172.21.15.150:0/3665642016 >> 172.21.15.150:0/3665642016 conn(0x7fa3c406b930 msgr2=0x7fa3c406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:32.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.203+0000 7fa3c3200640 1 -- 172.21.15.150:0/3665642016 shutdown_connections 2024-04-15T15:24:32.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.203+0000 7fa3c3200640 1 -- 172.21.15.150:0/3665642016 wait complete. 2024-04-15T15:24:32.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.204+0000 7fa3c3200640 1 Processor -- start 2024-04-15T15:24:32.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.204+0000 7fa3c3200640 1 -- start start 2024-04-15T15:24:32.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c4071f50 0x7fa3c414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c40cee00 0x7fa3c41582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c415cbe0 0x7fa3c4155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c406e3a0 con 0x7fa3c40cee00 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c406e0c0 con 0x7fa3c415cbe0 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c415dda0 con 0x7fa3c4071f50 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c1400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c40cee00 0x7fa3c41582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c1400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c40cee00 0x7fa3c41582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36008/0 (socket says 172.21.15.150:36008) 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.205+0000 7fa3c1400640 1 -- 172.21.15.150:0/38008779 learned_addr learned my addr 172.21.15.150:0/38008779 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3c1e00640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c4071f50 0x7fa3c414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3c2800640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c415cbe0 0x7fa3c4155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3c1400640 1 -- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c4071f50 msgr2=0x7fa3c414efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3c1400640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c4071f50 0x7fa3c414efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3c1400640 1 -- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c415cbe0 msgr2=0x7fa3c4155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3c1400640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c415cbe0 0x7fa3c4155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3c1400640 1 -- 172.21.15.150:0/38008779 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c415f850 con 0x7fa3c40cee00 2024-04-15T15:24:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3c1400640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c40cee00 0x7fa3c41582f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac02b930 tx=0x7fa3ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.206+0000 7fa3ba000640 1 -- 172.21.15.150:0/38008779 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac040070 con 0x7fa3c40cee00 2024-04-15T15:24:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.207+0000 7fa3ba000640 1 -- 172.21.15.150:0/38008779 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ac0399f0 con 0x7fa3c40cee00 2024-04-15T15:24:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.207+0000 7fa3ba000640 1 -- 172.21.15.150:0/38008779 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac035040 con 0x7fa3c40cee00 2024-04-15T15:24:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.207+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c415fb30 con 0x7fa3c40cee00 2024-04-15T15:24:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.207+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c4153080 con 0x7fa3c40cee00 2024-04-15T15:24:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.208+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c40755c0 con 0x7fa3c40cee00 2024-04-15T15:24:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.209+0000 7fa3ba000640 1 -- 172.21.15.150:0/38008779 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3ac038020 con 0x7fa3c40cee00 2024-04-15T15:24:32.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.209+0000 7fa3ba000640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa3a006c8f0 0x7fa3a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.210+0000 7fa3ba000640 1 -- 172.21.15.150:0/38008779 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa3ac0ad580 con 0x7fa3c40cee00 2024-04-15T15:24:32.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.210+0000 7fa3c1e00640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa3a006c8f0 0x7fa3a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.211+0000 7fa3c1e00640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa3a006c8f0 0x7fa3a006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006fd0 tx=0x7fa3b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:32.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.215+0000 7fa3ba000640 1 -- 172.21.15.150:0/38008779 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3ac07b9a0 con 0x7fa3c40cee00 2024-04-15T15:24:32.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.416+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3c40cdc30 con 0x7fa3c40cee00 2024-04-15T15:24:32.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.419+0000 7fa3ba000640 1 -- 172.21.15.150:0/38008779 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7fa3ac07b2a0 con 0x7fa3c40cee00 2024-04-15T15:24:32.421 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:32.421 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:32.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa3a006c8f0 msgr2=0x7fa3a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa3a006c8f0 0x7fa3a006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006fd0 tx=0x7fa3b4008040 comp rx=0 tx=0).stop 2024-04-15T15:24:32.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c40cee00 msgr2=0x7fa3c41582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c40cee00 0x7fa3c41582f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac02b930 tx=0x7fa3ac0046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 shutdown_connections 2024-04-15T15:24:32.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3c415cbe0 0x7fa3c4155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c40cee00 0x7fa3c41582f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa3a006c8f0 0x7fa3a006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 --2- 172.21.15.150:0/38008779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3c4071f50 0x7fa3c414efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.421+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 >> 172.21.15.150:0/38008779 conn(0x7fa3c406b930 msgr2=0x7fa3c4074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:32.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.422+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 shutdown_connections 2024-04-15T15:24:32.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.422+0000 7fa3c3200640 1 -- 172.21.15.150:0/38008779 wait complete. 2024-04-15T15:24:32.442 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:32.443 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:32.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.643+0000 7f5042000640 1 Processor -- start 2024-04-15T15:24:32.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.643+0000 7f5042000640 1 -- start start 2024-04-15T15:24:32.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.644+0000 7f5042000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f503c145170 0x7f503c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.644+0000 7f5042000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c0ce070 0x7f503c0b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.644+0000 7f5042000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f503c0b4410 con 0x7f503c0b59f0 2024-04-15T15:24:32.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.644+0000 7f5042000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f503c145550 con 0x7f503c145170 2024-04-15T15:24:32.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.645+0000 7f5042000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f503c0b46f0 con 0x7f503c0ce070 2024-04-15T15:24:32.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.645+0000 7f503b400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f503c145170 0x7f503c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.645+0000 7f5033e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c0ce070 0x7f503c0b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.645+0000 7f503be00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f503c0b59f0 0x7f503c06f260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:59816/0 (socket says 172.21.15.150:59816) 2024-04-15T15:24:32.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.646+0000 7f5033e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c0ce070 0x7f503c0b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:47536/0 (socket says 172.21.15.150:47536) 2024-04-15T15:24:32.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.646+0000 7f503be00640 1 -- 172.21.15.150:0/2434848451 learned_addr learned my addr 172.21.15.150:0/2434848451 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:32.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.646+0000 7f5033e00640 1 -- 172.21.15.150:0/2434848451 >> v2:172.21.15.187:3300/0 conn(0x7f503c145170 msgr2=0x7f503c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:32.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.646+0000 7f5033e00640 1 --2- 172.21.15.150:0/2434848451 >> v2:172.21.15.187:3300/0 conn(0x7f503c145170 0x7f503c0cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.646+0000 7f5033e00640 1 -- 172.21.15.150:0/2434848451 >> v1:172.21.15.187:6789/0 conn(0x7f503c0b59f0 legacy=0x7f503c06f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.646+0000 7f5033e00640 1 -- 172.21.15.150:0/2434848451 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c0b49d0 con 0x7f503c0ce070 2024-04-15T15:24:32.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.648+0000 7f5033e00640 1 --2- 172.21.15.150:0/2434848451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c0ce070 0x7f503c0b54b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50240097a0 tx=0x7f502402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe453ca0ee240ff server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:32.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.648+0000 7f503aa00640 1 -- 172.21.15.150:0/2434848451 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024035020 con 0x7f503c0ce070 2024-04-15T15:24:32.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.648+0000 7f503aa00640 1 -- 172.21.15.150:0/2434848451 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5024031810 con 0x7f503c0ce070 2024-04-15T15:24:32.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.648+0000 7f503aa00640 1 -- 172.21.15.150:0/2434848451 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024031be0 con 0x7f503c0ce070 2024-04-15T15:24:32.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.649+0000 7f5042000640 1 -- 172.21.15.150:0/2434848451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c0ce070 msgr2=0x7f503c0b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.649+0000 7f5042000640 1 --2- 172.21.15.150:0/2434848451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c0ce070 0x7f503c0b54b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50240097a0 tx=0x7f502402b8b0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.649+0000 7f5042000640 1 -- 172.21.15.150:0/2434848451 shutdown_connections 2024-04-15T15:24:32.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.649+0000 7f5042000640 1 --2- 172.21.15.150:0/2434848451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c0ce070 0x7f503c0b54b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.649+0000 7f5042000640 1 --2- 172.21.15.150:0/2434848451 >> v2:172.21.15.187:3300/0 conn(0x7f503c145170 0x7f503c0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.649+0000 7f5042000640 1 -- 172.21.15.150:0/2434848451 >> 172.21.15.150:0/2434848451 conn(0x7f503c0b2460 msgr2=0x7f503c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:32.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.649+0000 7f5042000640 1 -- 172.21.15.150:0/2434848451 shutdown_connections 2024-04-15T15:24:32.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.649+0000 7f5042000640 1 -- 172.21.15.150:0/2434848451 wait complete. 2024-04-15T15:24:32.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.651+0000 7f5042000640 1 Processor -- start 2024-04-15T15:24:32.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.651+0000 7f5042000640 1 -- start start 2024-04-15T15:24:32.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5042000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f503c0b59f0 0x7f503c15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5042000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f503c0ce070 0x7f503c1641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5042000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c145170 0x7f503c1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5042000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f503c0b4030 con 0x7f503c0ce070 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5042000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f503c0b4f90 con 0x7f503c0b59f0 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5042000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f503c0b41b0 con 0x7f503c145170 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5033e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f503c0ce070 0x7f503c1641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5033e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f503c0ce070 0x7f503c1641b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36020/0 (socket says 172.21.15.150:36020) 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f5033e00640 1 -- 172.21.15.150:0/339877436 learned_addr learned my addr 172.21.15.150:0/339877436 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f503b400640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f503c0b59f0 0x7f503c15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.652+0000 7f503be00640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c145170 0x7f503c1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033e00640 1 -- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c145170 msgr2=0x7f503c1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033e00640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c145170 0x7f503c1622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033e00640 1 -- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f503c0b59f0 msgr2=0x7f503c15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033e00640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f503c0b59f0 0x7f503c15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033e00640 1 -- 172.21.15.150:0/339877436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c164b70 con 0x7f503c0ce070 2024-04-15T15:24:32.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033e00640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f503c0ce070 0x7f503c1641b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f502402bec0 tx=0x7f50240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:32.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033400640 1 -- 172.21.15.150:0/339877436 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024035040 con 0x7f503c0ce070 2024-04-15T15:24:32.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033400640 1 -- 172.21.15.150:0/339877436 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502403ad00 con 0x7f503c0ce070 2024-04-15T15:24:32.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5033400640 1 -- 172.21.15.150:0/339877436 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50240466a0 con 0x7f503c0ce070 2024-04-15T15:24:32.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.653+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503c164e50 con 0x7f503c0ce070 2024-04-15T15:24:32.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.654+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503c165410 con 0x7f503c0ce070 2024-04-15T15:24:32.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.655+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503c068c80 con 0x7f503c0ce070 2024-04-15T15:24:32.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.656+0000 7f5033400640 1 -- 172.21.15.150:0/339877436 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5024038020 con 0x7f503c0ce070 2024-04-15T15:24:32.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.657+0000 7f5033400640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f501806c8f0 0x7f501806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:32.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.657+0000 7f5033400640 1 -- 172.21.15.150:0/339877436 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f50240adc20 con 0x7f503c0ce070 2024-04-15T15:24:32.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.658+0000 7f503b400640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f501806c8f0 0x7f501806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:32.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.659+0000 7f503b400640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f501806c8f0 0x7f501806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f503c0cfba0 tx=0x7f502c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:32.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.661+0000 7f5033400640 1 -- 172.21.15.150:0/339877436 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f502407c0e0 con 0x7f503c0ce070 2024-04-15T15:24:32.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.846+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f503c169080 con 0x7f503c0ce070 2024-04-15T15:24:32.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.847+0000 7f5033400640 1 -- 172.21.15.150:0/339877436 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f502407b9e0 con 0x7f503c0ce070 2024-04-15T15:24:32.849 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:32.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f501806c8f0 msgr2=0x7f501806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f501806c8f0 0x7f501806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f503c0cfba0 tx=0x7f502c008040 comp rx=0 tx=0).stop 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f503c0ce070 msgr2=0x7f503c1641b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f503c0ce070 0x7f503c1641b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f502402bec0 tx=0x7f50240046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 shutdown_connections 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f501806c8f0 0x7f501806edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f503c145170 0x7f503c1622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f503c0ce070 0x7f503c1641b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 --2- 172.21.15.150:0/339877436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f503c0b59f0 0x7f503c15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.849+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 >> 172.21.15.150:0/339877436 conn(0x7f503c0b2460 msgr2=0x7f503c06e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.850+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 shutdown_connections 2024-04-15T15:24:32.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:32.850+0000 7f5042000640 1 -- 172.21.15.150:0/339877436 wait complete. 2024-04-15T15:24:32.870 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:33.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.073+0000 7f7552a00640 1 Processor -- start 2024-04-15T15:24:33.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.073+0000 7f7552a00640 1 -- start start 2024-04-15T15:24:33.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.074+0000 7f7552a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f754c0725f0 0x7f754c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.074+0000 7f7552a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f754c0ce9e0 0x7f754c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7552a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f754c0cd2d0 con 0x7f754c1461c0 2024-04-15T15:24:33.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7552a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f754c0cf1d0 con 0x7f754c0725f0 2024-04-15T15:24:33.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7552a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f754c0cd080 con 0x7f754c0ce9e0 2024-04-15T15:24:33.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7550c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f754c0725f0 0x7f754c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7551600640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f754c1461c0 0x7f754c071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44864/0 (socket says 172.21.15.150:44864) 2024-04-15T15:24:33.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7551600640 1 -- 172.21.15.150:0/82672308 learned_addr learned my addr 172.21.15.150:0/82672308 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:33.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7552000640 1 --2- 172.21.15.150:0/82672308 >> v2:172.21.15.187:3300/0 conn(0x7f754c0ce9e0 0x7f754c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7550c00640 1 -- 172.21.15.150:0/82672308 >> v2:172.21.15.187:3300/0 conn(0x7f754c0ce9e0 msgr2=0x7f754c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:33.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7550c00640 1 --2- 172.21.15.150:0/82672308 >> v2:172.21.15.187:3300/0 conn(0x7f754c0ce9e0 0x7f754c06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f754be00640 1 -- 172.21.15.150:0/82672308 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1943884326 0 0) 0x7f754c0cd2d0 con 0x7f754c1461c0 2024-04-15T15:24:33.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7550c00640 1 -- 172.21.15.150:0/82672308 >> v1:172.21.15.150:6789/0 conn(0x7f754c1461c0 legacy=0x7f754c071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.075+0000 7f7550c00640 1 -- 172.21.15.150:0/82672308 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c0cd5b0 con 0x7f754c0725f0 2024-04-15T15:24:33.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f7550c00640 1 --2- 172.21.15.150:0/82672308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0ce4a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f754c0b60e0 tx=0x7f753802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1e8b69762bfb9a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f754be00640 1 -- 172.21.15.150:0/82672308 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538035020 con 0x7f754c0725f0 2024-04-15T15:24:33.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f754be00640 1 -- 172.21.15.150:0/82672308 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7538031830 con 0x7f754c0725f0 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f754be00640 1 -- 172.21.15.150:0/82672308 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538031c00 con 0x7f754c0725f0 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f7552a00640 1 -- 172.21.15.150:0/82672308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 msgr2=0x7f754c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f7552a00640 1 --2- 172.21.15.150:0/82672308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0ce4a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f754c0b60e0 tx=0x7f753802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f7552a00640 1 -- 172.21.15.150:0/82672308 shutdown_connections 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f7552a00640 1 --2- 172.21.15.150:0/82672308 >> v2:172.21.15.187:3300/0 conn(0x7f754c0ce9e0 0x7f754c06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f7552a00640 1 --2- 172.21.15.150:0/82672308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0ce4a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.077+0000 7f7552a00640 1 -- 172.21.15.150:0/82672308 >> 172.21.15.150:0/82672308 conn(0x7f754c06b930 msgr2=0x7f754c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.078+0000 7f7552a00640 1 -- 172.21.15.150:0/82672308 shutdown_connections 2024-04-15T15:24:33.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.078+0000 7f7552a00640 1 -- 172.21.15.150:0/82672308 wait complete. 2024-04-15T15:24:33.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.079+0000 7f7552a00640 1 Processor -- start 2024-04-15T15:24:33.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.079+0000 7f7552a00640 1 -- start start 2024-04-15T15:24:33.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7552a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7552a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f754c0ce9e0 0x7f754c1451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7552a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f754c1461c0 0x7f754c140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7552a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0cee30 con 0x7f754c0725f0 2024-04-15T15:24:33.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7552a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0cdb70 con 0x7f754c1461c0 2024-04-15T15:24:33.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7552a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f754c0cefb0 con 0x7f754c0ce9e0 2024-04-15T15:24:33.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7551600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7551600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36044/0 (socket says 172.21.15.150:36044) 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.080+0000 7f7551600640 1 -- 172.21.15.150:0/787898452 learned_addr learned my addr 172.21.15.150:0/787898452 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7551600640 1 -- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f754c0ce9e0 msgr2=0x7f754c1451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7551600640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f754c0ce9e0 0x7f754c1451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7551600640 1 -- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f754c1461c0 msgr2=0x7f754c140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7551600640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f754c1461c0 0x7f754c140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7551600640 1 -- 172.21.15.150:0/787898452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c0b0770 con 0x7f754c0725f0 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7551600640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0aff20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f754000c430 tx=0x7f754000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7549600640 1 -- 172.21.15.150:0/787898452 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75400074e0 con 0x7f754c0725f0 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7549600640 1 -- 172.21.15.150:0/787898452 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75400047e0 con 0x7f754c0725f0 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c0b0a50 con 0x7f754c0725f0 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7549600640 1 -- 172.21.15.150:0/787898452 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540004bd0 con 0x7f754c0725f0 2024-04-15T15:24:33.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.081+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c0b1010 con 0x7f754c0725f0 2024-04-15T15:24:33.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.082+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c0755c0 con 0x7f754c0725f0 2024-04-15T15:24:33.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.083+0000 7f7549600640 1 -- 172.21.15.150:0/787898452 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7540011070 con 0x7f754c0725f0 2024-04-15T15:24:33.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.085+0000 7f7549600640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.085+0000 7f7549600640 1 -- 172.21.15.150:0/787898452 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f754008b7a0 con 0x7f754c0725f0 2024-04-15T15:24:33.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.085+0000 7f7550c00640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.086+0000 7f7550c00640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f752c06c8f0 0x7f752c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7538036000 tx=0x7f7538030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.089+0000 7f7549600640 1 -- 172.21.15.150:0/787898452 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7540059c60 con 0x7f754c0725f0 2024-04-15T15:24:33.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.274+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f754c06f910 con 0x7f754c0725f0 2024-04-15T15:24:33.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.275+0000 7f7549600640 1 -- 172.21.15.150:0/787898452 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f7540059560 con 0x7f754c0725f0 2024-04-15T15:24:33.277 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:33.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f752c06c8f0 msgr2=0x7f752c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f752c06c8f0 0x7f752c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7538036000 tx=0x7f7538030000 comp rx=0 tx=0).stop 2024-04-15T15:24:33.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 msgr2=0x7f754c0aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0aff20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f754000c430 tx=0x7f754000c8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 shutdown_connections 2024-04-15T15:24:33.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f754c1461c0 0x7f754c140680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f754c0ce9e0 0x7f754c1451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 --2- 172.21.15.150:0/787898452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f754c0725f0 0x7f754c0aff20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 >> 172.21.15.150:0/787898452 conn(0x7f754c06b930 msgr2=0x7f754c0b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:33.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.278+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 shutdown_connections 2024-04-15T15:24:33.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.279+0000 7f7552a00640 1 -- 172.21.15.150:0/787898452 wait complete. 2024-04-15T15:24:33.299 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:33.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.508+0000 7f94bdc00640 1 Processor -- start 2024-04-15T15:24:33.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.508+0000 7f94bdc00640 1 -- start start 2024-04-15T15:24:33.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.510+0000 7f94bdc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f94b80d18d0 0x7f94b80d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.510+0000 7f94bdc00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b815f130 con 0x7f94b815ed50 2024-04-15T15:24:33.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.510+0000 7f94bdc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b80d3210 con 0x7f94b80cf1a0 2024-04-15T15:24:33.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.510+0000 7f94bdc00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f94b80b7e40 con 0x7f94b80d18d0 2024-04-15T15:24:33.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.510+0000 7f94afe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f94b80d18d0 0x7f94b80d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.510+0000 7f94afe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f94b80d18d0 0x7f94b80d1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36056/0 (socket says 172.21.15.150:36056) 2024-04-15T15:24:33.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.510+0000 7f94afe00640 1 -- 172.21.15.150:0/2061679580 learned_addr learned my addr 172.21.15.150:0/2061679580 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:33.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.510+0000 7f94b6a00640 1 --1- 172.21.15.150:0/2061679580 >> v1:172.21.15.150:6789/0 conn(0x7f94b815ed50 0x7f94b80d2b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44868/0 (socket says 172.21.15.150:44868) 2024-04-15T15:24:33.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.511+0000 7f94afe00640 1 -- 172.21.15.150:0/2061679580 >> v1:172.21.15.187:6789/0 conn(0x7f94b80cf1a0 legacy=0x7f94b80cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:33.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.511+0000 7f94afe00640 1 -- 172.21.15.150:0/2061679580 >> v1:172.21.15.150:6789/0 conn(0x7f94b815ed50 legacy=0x7f94b80d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.511+0000 7f94afe00640 1 -- 172.21.15.150:0/2061679580 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b80d34e0 con 0x7f94b80d18d0 2024-04-15T15:24:33.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.511+0000 7f94b6000640 1 -- 172.21.15.150:0/2061679580 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410056483 0 0) 0x7f94b815f130 con 0x7f94b815ed50 2024-04-15T15:24:33.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.512+0000 7f94afe00640 1 --2- 172.21.15.150:0/2061679580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b80d18d0 0x7f94b80d1d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009870 tx=0x7f94a002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d39f1b479a878e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.512+0000 7f94b6000640 1 -- 172.21.15.150:0/2061679580 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035020 con 0x7f94b80d18d0 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.512+0000 7f94b6000640 1 -- 172.21.15.150:0/2061679580 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a002d040 con 0x7f94b80d18d0 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.512+0000 7f94b6000640 1 -- 172.21.15.150:0/2061679580 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0031a80 con 0x7f94b80d18d0 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.513+0000 7f94bdc00640 1 -- 172.21.15.150:0/2061679580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b80d18d0 msgr2=0x7f94b80d1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.513+0000 7f94bdc00640 1 --2- 172.21.15.150:0/2061679580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b80d18d0 0x7f94b80d1d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009870 tx=0x7f94a002b900 comp rx=0 tx=0).stop 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.513+0000 7f94bdc00640 1 -- 172.21.15.150:0/2061679580 shutdown_connections 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.513+0000 7f94bdc00640 1 --2- 172.21.15.150:0/2061679580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b80d18d0 0x7f94b80d1d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.513+0000 7f94bdc00640 1 -- 172.21.15.150:0/2061679580 >> 172.21.15.150:0/2061679580 conn(0x7f94b806d400 msgr2=0x7f94b806d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.513+0000 7f94bdc00640 1 -- 172.21.15.150:0/2061679580 shutdown_connections 2024-04-15T15:24:33.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.513+0000 7f94bdc00640 1 -- 172.21.15.150:0/2061679580 wait complete. 2024-04-15T15:24:33.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.514+0000 7f94bdc00640 1 Processor -- start 2024-04-15T15:24:33.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.515+0000 7f94bdc00640 1 -- start start 2024-04-15T15:24:33.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.515+0000 7f94bdc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94b80cf1a0 0x7f94b81f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94bdc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94b80d18d0 0x7f94b814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94bdc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b815ed50 0x7f94b80cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94bdc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f94b80b79d0 con 0x7f94b815ed50 2024-04-15T15:24:33.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94bdc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94b80d3aa0 con 0x7f94b80cf1a0 2024-04-15T15:24:33.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94bdc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94b80b7b50 con 0x7f94b80d18d0 2024-04-15T15:24:33.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b815ed50 0x7f94b80cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b815ed50 0x7f94b80cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36062/0 (socket says 172.21.15.150:36062) 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b7400640 1 -- 172.21.15.150:0/90204663 learned_addr learned my addr 172.21.15.150:0/90204663 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b6a00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94b80cf1a0 0x7f94b81f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94afe00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94b80d18d0 0x7f94b814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b7400640 1 -- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94b80d18d0 msgr2=0x7f94b814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b7400640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94b80d18d0 0x7f94b814dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b7400640 1 -- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94b80cf1a0 msgr2=0x7f94b81f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b7400640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94b80cf1a0 0x7f94b81f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b7400640 1 -- 172.21.15.150:0/90204663 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b814e2b0 con 0x7f94b815ed50 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.516+0000 7f94b6a00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94b80cf1a0 0x7f94b81f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:33.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.517+0000 7f94b7400640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b815ed50 0x7f94b80cdbe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c780 tx=0x7f94b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.517+0000 7f94aea00640 1 -- 172.21.15.150:0/90204663 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0004b60 con 0x7f94b815ed50 2024-04-15T15:24:33.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.517+0000 7f94aea00640 1 -- 172.21.15.150:0/90204663 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b0004d00 con 0x7f94b815ed50 2024-04-15T15:24:33.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.517+0000 7f94aea00640 1 -- 172.21.15.150:0/90204663 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b000aa90 con 0x7f94b815ed50 2024-04-15T15:24:33.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.517+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b80c9170 con 0x7f94b815ed50 2024-04-15T15:24:33.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.517+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b80c9430 con 0x7f94b815ed50 2024-04-15T15:24:33.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.518+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b8068c80 con 0x7f94b815ed50 2024-04-15T15:24:33.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.519+0000 7f94aea00640 1 -- 172.21.15.150:0/90204663 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94b000ac30 con 0x7f94b815ed50 2024-04-15T15:24:33.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.520+0000 7f94aea00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.520+0000 7f94aea00640 1 -- 172.21.15.150:0/90204663 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f94b008be00 con 0x7f94b815ed50 2024-04-15T15:24:33.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.521+0000 7f94b6a00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.522+0000 7f94b6a00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f949406c8f0 0x7f949406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f94a8000f60 tx=0x7f94a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.525+0000 7f94aea00640 1 -- 172.21.15.150:0/90204663 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94b0020070 con 0x7f94b815ed50 2024-04-15T15:24:33.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.724+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94b80cff90 con 0x7f94b815ed50 2024-04-15T15:24:33.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.725+0000 7f94aea00640 1 -- 172.21.15.150:0/90204663 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f94b814e2b0 con 0x7f94b815ed50 2024-04-15T15:24:33.727 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:33.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f949406c8f0 msgr2=0x7f949406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f949406c8f0 0x7f949406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f94a8000f60 tx=0x7f94a8009040 comp rx=0 tx=0).stop 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b815ed50 msgr2=0x7f94b80cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b815ed50 0x7f94b80cdbe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c780 tx=0x7f94b000cc40 comp rx=0 tx=0).stop 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 shutdown_connections 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94b815ed50 0x7f94b80cdbe0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94b80d18d0 0x7f94b814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 --2- 172.21.15.150:0/90204663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94b80cf1a0 0x7f94b81f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 >> 172.21.15.150:0/90204663 conn(0x7f94b806d400 msgr2=0x7f94b80b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 shutdown_connections 2024-04-15T15:24:33.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.728+0000 7f94bdc00640 1 -- 172.21.15.150:0/90204663 wait complete. 2024-04-15T15:24:33.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:33.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.940+0000 7f125d000640 1 Processor -- start 2024-04-15T15:24:33.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.941+0000 7f125d000640 1 -- start start 2024-04-15T15:24:33.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.942+0000 7f125d000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1258162020 0x7f1258162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.942+0000 7f125d000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f12580cf110 0x7f12580d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.942+0000 7f125d000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12580d0090 con 0x7f12580d1ac0 2024-04-15T15:24:33.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.942+0000 7f125d000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f12580d1f50 con 0x7f12580cf110 2024-04-15T15:24:33.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.942+0000 7f125d000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1258163440 con 0x7f1258162020 2024-04-15T15:24:33.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.942+0000 7f124fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f12580cf110 0x7f12580d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.942+0000 7f124fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f12580cf110 0x7f12580d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36070/0 (socket says 172.21.15.150:36070) 2024-04-15T15:24:33.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.942+0000 7f124fe00640 1 -- 172.21.15.150:0/3956915142 learned_addr learned my addr 172.21.15.150:0/3956915142 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:33.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.943+0000 7f1257400640 1 --2- 172.21.15.150:0/3956915142 >> v2:172.21.15.187:3300/0 conn(0x7f1258162020 0x7f1258162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.943+0000 7f124fe00640 1 -- 172.21.15.150:0/3956915142 >> v2:172.21.15.187:3300/0 conn(0x7f1258162020 msgr2=0x7f1258162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:33.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.943+0000 7f124fe00640 1 --2- 172.21.15.150:0/3956915142 >> v2:172.21.15.187:3300/0 conn(0x7f1258162020 0x7f1258162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.943+0000 7f124fe00640 1 -- 172.21.15.150:0/3956915142 >> v1:172.21.15.187:6789/0 conn(0x7f12580d1ac0 legacy=0x7f12580b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:33.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.943+0000 7f124fe00640 1 -- 172.21.15.150:0/3956915142 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12580cfad0 con 0x7f12580cf110 2024-04-15T15:24:33.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.943+0000 7f1257400640 1 --2- 172.21.15.150:0/3956915142 >> v2:172.21.15.187:3300/0 conn(0x7f1258162020 0x7f1258162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:33.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.944+0000 7f124fe00640 1 --2- 172.21.15.150:0/3956915142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12580cf110 0x7f12580d1580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f125808cd90 tx=0x7f124002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1197520004b407b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.944+0000 7f1256a00640 1 -- 172.21.15.150:0/3956915142 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240035020 con 0x7f12580cf110 2024-04-15T15:24:33.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.944+0000 7f1256a00640 1 -- 172.21.15.150:0/3956915142 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240031870 con 0x7f12580cf110 2024-04-15T15:24:33.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.944+0000 7f1256a00640 1 -- 172.21.15.150:0/3956915142 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240031c40 con 0x7f12580cf110 2024-04-15T15:24:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.945+0000 7f125d000640 1 -- 172.21.15.150:0/3956915142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12580cf110 msgr2=0x7f12580d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.945+0000 7f125d000640 1 --2- 172.21.15.150:0/3956915142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12580cf110 0x7f12580d1580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f125808cd90 tx=0x7f124002b910 comp rx=0 tx=0).stop 2024-04-15T15:24:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.945+0000 7f125d000640 1 -- 172.21.15.150:0/3956915142 shutdown_connections 2024-04-15T15:24:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.945+0000 7f125d000640 1 --2- 172.21.15.150:0/3956915142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12580cf110 0x7f12580d1580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.945+0000 7f125d000640 1 --2- 172.21.15.150:0/3956915142 >> v2:172.21.15.187:3300/0 conn(0x7f1258162020 0x7f1258162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.945+0000 7f125d000640 1 -- 172.21.15.150:0/3956915142 >> 172.21.15.150:0/3956915142 conn(0x7f125806cf00 msgr2=0x7f125806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.945+0000 7f125d000640 1 -- 172.21.15.150:0/3956915142 shutdown_connections 2024-04-15T15:24:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.945+0000 7f125d000640 1 -- 172.21.15.150:0/3956915142 wait complete. 2024-04-15T15:24:33.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.947+0000 7f125d000640 1 Processor -- start 2024-04-15T15:24:33.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.947+0000 7f125d000640 1 -- start start 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f125d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12580cf110 0x7f125814ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f125d000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12580d1ac0 0x7f12580c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f125d000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1258162020 0x7f12580c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f125d000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12580cf880 con 0x7f1258162020 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f125d000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12580b7b30 con 0x7f12580d1ac0 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f125d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12580d2b90 con 0x7f12580cf110 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f1257e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1258162020 0x7f12580c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f1257e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1258162020 0x7f12580c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36072/0 (socket says 172.21.15.150:36072) 2024-04-15T15:24:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.948+0000 7f1257e00640 1 -- 172.21.15.150:0/1030960516 learned_addr learned my addr 172.21.15.150:0/1030960516 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f1257400640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12580cf110 0x7f125814ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f124fe00640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12580d1ac0 0x7f12580c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f1257e00640 1 -- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12580cf110 msgr2=0x7f125814ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f1257e00640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12580cf110 0x7f125814ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f1257e00640 1 -- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12580d1ac0 msgr2=0x7f12580c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f1257e00640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12580d1ac0 0x7f12580c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f1257e00640 1 -- 172.21.15.150:0/1030960516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125814f130 con 0x7f1258162020 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f1257400640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12580cf110 0x7f125814ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f1257e00640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1258162020 0x7f12580c8890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f125000c780 tx=0x7f125000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f124f400640 1 -- 172.21.15.150:0/1030960516 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250004b60 con 0x7f1258162020 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f124f400640 1 -- 172.21.15.150:0/1030960516 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1250004d00 con 0x7f1258162020 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f124f400640 1 -- 172.21.15.150:0/1030960516 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125000aa30 con 0x7f1258162020 2024-04-15T15:24:33.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.949+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12580ca5a0 con 0x7f1258162020 2024-04-15T15:24:33.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.950+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12580cbb40 con 0x7f1258162020 2024-04-15T15:24:33.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.951+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12580755c0 con 0x7f1258162020 2024-04-15T15:24:33.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.952+0000 7f124f400640 1 -- 172.21.15.150:0/1030960516 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f125000abd0 con 0x7f1258162020 2024-04-15T15:24:33.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.952+0000 7f124f400640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f123406c920 0x7f123406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:33.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.953+0000 7f124f400640 1 -- 172.21.15.150:0/1030960516 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f125008be30 con 0x7f1258162020 2024-04-15T15:24:33.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.953+0000 7f1257400640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f123406c920 0x7f123406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:33.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.954+0000 7f1257400640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f123406c920 0x7f123406ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1248006fd0 tx=0x7f1248008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:33.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:33.957+0000 7f124f400640 1 -- 172.21.15.150:0/1030960516 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1250020070 con 0x7f1258162020 2024-04-15T15:24:34.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.159+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12580b7650 con 0x7f1258162020 2024-04-15T15:24:34.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.160+0000 7f124f400640 1 -- 172.21.15.150:0/1030960516 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16653 (secure 0 0 0) 0x7f125005a1c0 con 0x7f1258162020 2024-04-15T15:24:34.162 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:34.163 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:23:56.248364+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f123406c920 msgr2=0x7f123406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f123406c920 0x7f123406ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1248006fd0 tx=0x7f1248008040 comp rx=0 tx=0).stop 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1258162020 msgr2=0x7f12580c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1258162020 0x7f12580c8890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f125000c780 tx=0x7f125000cc40 comp rx=0 tx=0).stop 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 shutdown_connections 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1258162020 0x7f12580c8890 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12580d1ac0 0x7f12580c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f123406c920 0x7f123406ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 --2- 172.21.15.150:0/1030960516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12580cf110 0x7f125814ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 >> 172.21.15.150:0/1030960516 conn(0x7f125806cf00 msgr2=0x7f125815e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:34.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 shutdown_connections 2024-04-15T15:24:34.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.163+0000 7f125d000640 1 -- 172.21.15.150:0/1030960516 wait complete. 2024-04-15T15:24:34.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:34.183 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:34.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.397+0000 7fcbaae00640 1 Processor -- start 2024-04-15T15:24:34.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.397+0000 7fcbaae00640 1 -- start start 2024-04-15T15:24:34.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.399+0000 7fcbaae00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcba415a440 0x7fcba415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.399+0000 7fcbaae00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcba4072150 0x7fcba4072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.399+0000 7fcbaae00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcba415d520 con 0x7fcba406fc80 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.399+0000 7fcbaae00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fcba406e280 con 0x7fcba4072150 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.399+0000 7fcbaae00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcba4072ad0 con 0x7fcba415a440 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.399+0000 7fcba8e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcba4072150 0x7fcba4072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.400+0000 7fcba8e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcba4072150 0x7fcba4072590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36074/0 (socket says 172.21.15.150:36074) 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.400+0000 7fcba8e00640 1 -- 172.21.15.150:0/3521012153 learned_addr learned my addr 172.21.15.150:0/3521012153 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.400+0000 7fcba3e00640 1 --2- 172.21.15.150:0/3521012153 >> v2:172.21.15.187:3300/0 conn(0x7fcba415a440 0x7fcba415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.400+0000 7fcba8e00640 1 -- 172.21.15.150:0/3521012153 >> v2:172.21.15.187:3300/0 conn(0x7fcba415a440 msgr2=0x7fcba415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.400+0000 7fcba8e00640 1 --2- 172.21.15.150:0/3521012153 >> v2:172.21.15.187:3300/0 conn(0x7fcba415a440 0x7fcba415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.400+0000 7fcba8e00640 1 -- 172.21.15.150:0/3521012153 >> v1:172.21.15.187:6789/0 conn(0x7fcba406fc80 legacy=0x7fcba40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:34.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.400+0000 7fcba8e00640 1 -- 172.21.15.150:0/3521012153 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba415d800 con 0x7fcba4072150 2024-04-15T15:24:34.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.401+0000 7fcba8e00640 1 --2- 172.21.15.150:0/3521012153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4072590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcba406c9a0 tx=0x7fcb9c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5ace59a7d92cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:34.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.401+0000 7fcba2a00640 1 -- 172.21.15.150:0/3521012153 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c02ca80 con 0x7fcba4072150 2024-04-15T15:24:34.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.401+0000 7fcba2a00640 1 -- 172.21.15.150:0/3521012153 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9c02cc20 con 0x7fcba4072150 2024-04-15T15:24:34.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.401+0000 7fcba2a00640 1 -- 172.21.15.150:0/3521012153 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0343f0 con 0x7fcba4072150 2024-04-15T15:24:34.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.402+0000 7fcbaae00640 1 -- 172.21.15.150:0/3521012153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 msgr2=0x7fcba4072590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.402+0000 7fcbaae00640 1 --2- 172.21.15.150:0/3521012153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4072590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcba406c9a0 tx=0x7fcb9c02bee0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.402+0000 7fcbaae00640 1 -- 172.21.15.150:0/3521012153 shutdown_connections 2024-04-15T15:24:34.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.402+0000 7fcbaae00640 1 --2- 172.21.15.150:0/3521012153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4072590 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.402+0000 7fcbaae00640 1 --2- 172.21.15.150:0/3521012153 >> v2:172.21.15.187:3300/0 conn(0x7fcba415a440 0x7fcba415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.402+0000 7fcbaae00640 1 -- 172.21.15.150:0/3521012153 >> 172.21.15.150:0/3521012153 conn(0x7fcba40b2460 msgr2=0x7fcba40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:34.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.402+0000 7fcbaae00640 1 -- 172.21.15.150:0/3521012153 shutdown_connections 2024-04-15T15:24:34.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.402+0000 7fcbaae00640 1 -- 172.21.15.150:0/3521012153 wait complete. 2024-04-15T15:24:34.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.404+0000 7fcbaae00640 1 Processor -- start 2024-04-15T15:24:34.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.404+0000 7fcbaae00640 1 -- start start 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcbaae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba406fc80 0x7fcba4158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcbaae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcbaae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba415a440 0x7fcba41421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcbaae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcba415a820 con 0x7fcba4072150 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcbaae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcba40cd7f0 con 0x7fcba415a440 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcbaae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcba415ddc0 con 0x7fcba406fc80 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcba3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcba3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4141c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36080/0 (socket says 172.21.15.150:36080) 2024-04-15T15:24:34.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.405+0000 7fcba3400640 1 -- 172.21.15.150:0/2159424427 learned_addr learned my addr 172.21.15.150:0/2159424427 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcba3e00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba406fc80 0x7fcba4158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcba8e00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba415a440 0x7fcba41421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcba3400640 1 -- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba406fc80 msgr2=0x7fcba4158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcba3400640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba406fc80 0x7fcba4158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcba3400640 1 -- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba415a440 msgr2=0x7fcba41421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcba3400640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba415a440 0x7fcba41421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcba3400640 1 -- 172.21.15.150:0/2159424427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba40b50a0 con 0x7fcba4072150 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcba3400640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4141c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c400 tx=0x7fcb9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcb9be00640 1 -- 172.21.15.150:0/2159424427 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90011790 con 0x7fcba4072150 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcb9be00640 1 -- 172.21.15.150:0/2159424427 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb90017070 con 0x7fcba4072150 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcb9be00640 1 -- 172.21.15.150:0/2159424427 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90011bb0 con 0x7fcba4072150 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.406+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba40b5380 con 0x7fcba4072150 2024-04-15T15:24:34.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.407+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba40adcd0 con 0x7fcba4072150 2024-04-15T15:24:34.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.408+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba4068c80 con 0x7fcba4072150 2024-04-15T15:24:34.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.409+0000 7fcb9be00640 1 -- 172.21.15.150:0/2159424427 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb9001c460 con 0x7fcba4072150 2024-04-15T15:24:34.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.410+0000 7fcb9be00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb8406c8f0 0x7fcb8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.410+0000 7fcb9be00640 1 -- 172.21.15.150:0/2159424427 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcb9008ba10 con 0x7fcba4072150 2024-04-15T15:24:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.410+0000 7fcba3e00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb8406c8f0 0x7fcb8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.411+0000 7fcba3e00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb8406c8f0 0x7fcb8406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006fd0 tx=0x7fcb94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:34.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.415+0000 7fcb9be00640 1 -- 172.21.15.150:0/2159424427 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb90059e30 con 0x7fcba4072150 2024-04-15T15:24:34.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.614+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcba40730e0 con 0x7fcba4072150 2024-04-15T15:24:34.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.615+0000 7fcb9be00640 1 -- 172.21.15.150:0/2159424427 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fcb90059730 con 0x7fcba4072150 2024-04-15T15:24:34.617 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:34.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb8406c8f0 msgr2=0x7fcb8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb8406c8f0 0x7fcb8406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006fd0 tx=0x7fcb94008040 comp rx=0 tx=0).stop 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 msgr2=0x7fcba4141c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4141c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c400 tx=0x7fcb9000c8c0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 shutdown_connections 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcba415a440 0x7fcba41421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4072150 0x7fcba4141c80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb8406c8f0 0x7fcb8406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 --2- 172.21.15.150:0/2159424427 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcba406fc80 0x7fcba4158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 >> 172.21.15.150:0/2159424427 conn(0x7fcba40b2460 msgr2=0x7fcba415ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 shutdown_connections 2024-04-15T15:24:34.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.618+0000 7fcbaae00640 1 -- 172.21.15.150:0/2159424427 wait complete. 2024-04-15T15:24:34.639 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:34.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.847+0000 7f4894a00640 1 Processor -- start 2024-04-15T15:24:34.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.847+0000 7f4894a00640 1 -- start start 2024-04-15T15:24:34.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.849+0000 7f4894a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f48900b8b30 0x7f48901696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.849+0000 7f4894a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48900d2c80 con 0x7f48900d16f0 2024-04-15T15:24:34.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.849+0000 7f4894a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48900d2f20 con 0x7f489015eb50 2024-04-15T15:24:34.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.849+0000 7f4894a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f48900d3850 con 0x7f48900b8b30 2024-04-15T15:24:34.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.849+0000 7f488e800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f48900b8b30 0x7f48901696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.849+0000 7f488d400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f48900d16f0 0x7f48900d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44884/0 (socket says 172.21.15.150:44884) 2024-04-15T15:24:34.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.849+0000 7f488d400640 1 -- 172.21.15.150:0/191873026 learned_addr learned my addr 172.21.15.150:0/191873026 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:34.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.850+0000 7f488ca00640 1 -- 172.21.15.150:0/191873026 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162954613 0 0) 0x7f48900d2c80 con 0x7f48900d16f0 2024-04-15T15:24:34.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.850+0000 7f488ca00640 1 -- 172.21.15.150:0/191873026 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4874003520 con 0x7f48900d16f0 2024-04-15T15:24:34.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.850+0000 7f488e800640 1 -- 172.21.15.150:0/191873026 >> v1:172.21.15.187:6789/0 conn(0x7f489015eb50 legacy=0x7f48900d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:34.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.850+0000 7f488e800640 1 -- 172.21.15.150:0/191873026 >> v1:172.21.15.150:6789/0 conn(0x7f48900d16f0 legacy=0x7f48900d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.850+0000 7f488e800640 1 -- 172.21.15.150:0/191873026 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48900d3200 con 0x7f48900b8b30 2024-04-15T15:24:34.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.851+0000 7f488e800640 1 --2- 172.21.15.150:0/191873026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900b8b30 0x7f48901696f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f48880097a0 tx=0x7f488802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b553addc3f51e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:34.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.851+0000 7f488ca00640 1 -- 172.21.15.150:0/191873026 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888035020 con 0x7f48900b8b30 2024-04-15T15:24:34.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.851+0000 7f488ca00640 1 -- 172.21.15.150:0/191873026 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4888031830 con 0x7f48900b8b30 2024-04-15T15:24:34.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.851+0000 7f488ca00640 1 -- 172.21.15.150:0/191873026 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888031c00 con 0x7f48900b8b30 2024-04-15T15:24:34.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.852+0000 7f4894a00640 1 -- 172.21.15.150:0/191873026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900b8b30 msgr2=0x7f48901696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.852+0000 7f4894a00640 1 --2- 172.21.15.150:0/191873026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900b8b30 0x7f48901696f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f48880097a0 tx=0x7f488802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.852+0000 7f4894a00640 1 -- 172.21.15.150:0/191873026 shutdown_connections 2024-04-15T15:24:34.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.852+0000 7f4894a00640 1 --2- 172.21.15.150:0/191873026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900b8b30 0x7f48901696f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.852+0000 7f4894a00640 1 -- 172.21.15.150:0/191873026 >> 172.21.15.150:0/191873026 conn(0x7f489006d1c0 msgr2=0x7f489006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:34.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.852+0000 7f4894a00640 1 -- 172.21.15.150:0/191873026 shutdown_connections 2024-04-15T15:24:34.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.852+0000 7f4894a00640 1 -- 172.21.15.150:0/191873026 wait complete. 2024-04-15T15:24:34.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.854+0000 7f4894a00640 1 Processor -- start 2024-04-15T15:24:34.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.854+0000 7f4894a00640 1 -- start start 2024-04-15T15:24:34.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f4894a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48900b8b30 0x7f48900b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f4894a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900d16f0 0x7f4890157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f4894a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f489015eb50 0x7f48900d48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f4894a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f48900cfed0 con 0x7f48900d16f0 2024-04-15T15:24:34.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f4894a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f489015ef30 con 0x7f48900b8b30 2024-04-15T15:24:34.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f4894a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48900d4520 con 0x7f489015eb50 2024-04-15T15:24:34.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f488d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900d16f0 0x7f4890157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f488d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900d16f0 0x7f4890157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36098/0 (socket says 172.21.15.150:36098) 2024-04-15T15:24:34.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f488d400640 1 -- 172.21.15.150:0/1832317614 learned_addr learned my addr 172.21.15.150:0/1832317614 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:34.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.855+0000 7f488de00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48900b8b30 0x7f48900b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f488e800640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f489015eb50 0x7f48900d48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f488d400640 1 -- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f489015eb50 msgr2=0x7f48900d48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f488d400640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f489015eb50 0x7f48900d48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f488d400640 1 -- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48900b8b30 msgr2=0x7f48900b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f488d400640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48900b8b30 0x7f48900b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f488d400640 1 -- 172.21.15.150:0/1832317614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48900b56b0 con 0x7f48900d16f0 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f488d400640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900d16f0 0x7f4890157f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f48780010e0 tx=0x7f487800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f4886000640 1 -- 172.21.15.150:0/1832317614 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878017070 con 0x7f48900d16f0 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f4886000640 1 -- 172.21.15.150:0/1832317614 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4878005d00 con 0x7f48900d16f0 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48900b5990 con 0x7f48900d16f0 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.856+0000 7f4886000640 1 -- 172.21.15.150:0/1832317614 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48780072b0 con 0x7f48900d16f0 2024-04-15T15:24:34.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.857+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48900b6230 con 0x7f48900d16f0 2024-04-15T15:24:34.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.858+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48900b5b60 con 0x7f48900d16f0 2024-04-15T15:24:34.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.859+0000 7f4886000640 1 -- 172.21.15.150:0/1832317614 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f487801a070 con 0x7f48900d16f0 2024-04-15T15:24:34.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.859+0000 7f4886000640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f486c06c8f0 0x7f486c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:34.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.859+0000 7f4886000640 1 -- 172.21.15.150:0/1832317614 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f487808af40 con 0x7f48900d16f0 2024-04-15T15:24:34.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.860+0000 7f488de00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f486c06c8f0 0x7f486c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:34.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.861+0000 7f488de00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f486c06c8f0 0x7f486c06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4880009f20 tx=0x7f48800094d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:34.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:34.864+0000 7f4886000640 1 -- 172.21.15.150:0/1832317614 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4878059400 con 0x7f48900d16f0 2024-04-15T15:24:35.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.048+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48900b6430 con 0x7f48900d16f0 2024-04-15T15:24:35.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.049+0000 7f4886000640 1 -- 172.21.15.150:0/1832317614 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f48900b6430 con 0x7f48900d16f0 2024-04-15T15:24:35.051 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:35.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f486c06c8f0 msgr2=0x7f486c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f486c06c8f0 0x7f486c06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4880009f20 tx=0x7f48800094d0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900d16f0 msgr2=0x7f4890157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900d16f0 0x7f4890157f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f48780010e0 tx=0x7f487800c960 comp rx=0 tx=0).stop 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 shutdown_connections 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f486c06c8f0 0x7f486c06edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f489015eb50 0x7f48900d48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48900d16f0 0x7f4890157f10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 --2- 172.21.15.150:0/1832317614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48900b8b30 0x7f48900b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 >> 172.21.15.150:0/1832317614 conn(0x7f489006d1c0 msgr2=0x7f489015de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 shutdown_connections 2024-04-15T15:24:35.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.052+0000 7f4894a00640 1 -- 172.21.15.150:0/1832317614 wait complete. 2024-04-15T15:24:35.072 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:35.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.284+0000 7f915f200640 1 Processor -- start 2024-04-15T15:24:35.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.284+0000 7f915f200640 1 -- start start 2024-04-15T15:24:35.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.286+0000 7f915f200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f91600725f0 0x7f91600ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.286+0000 7f915f200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f91600ce9e0 0x7f916006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.286+0000 7f915f200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91600cd2d0 con 0x7f91601461c0 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.286+0000 7f915f200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f91600cf1d0 con 0x7f91600ce9e0 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.286+0000 7f915f200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f91600cd080 con 0x7f91600725f0 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.286+0000 7f915de00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f91601461c0 0x7f9160071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:44896/0 (socket says 172.21.15.150:44896) 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.286+0000 7f915de00640 1 -- 172.21.15.150:0/2425762157 learned_addr learned my addr 172.21.15.150:0/2425762157 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.286+0000 7f915e800640 1 --2- 172.21.15.150:0/2425762157 >> v2:172.21.15.150:3300/0 conn(0x7f91600ce9e0 0x7f916006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.287+0000 7f9157e00640 1 --2- 172.21.15.150:0/2425762157 >> v2:172.21.15.187:3300/0 conn(0x7f91600725f0 0x7f91600ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.287+0000 7f915d400640 1 -- 172.21.15.150:0/2425762157 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114128556 0 0) 0x7f91600cd2d0 con 0x7f91601461c0 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.287+0000 7f915d400640 1 -- 172.21.15.150:0/2425762157 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9144003520 con 0x7f91601461c0 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.287+0000 7f915e800640 1 -- 172.21.15.150:0/2425762157 >> v2:172.21.15.187:3300/0 conn(0x7f91600725f0 msgr2=0x7f91600ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.287+0000 7f915e800640 1 --2- 172.21.15.150:0/2425762157 >> v2:172.21.15.187:3300/0 conn(0x7f91600725f0 0x7f91600ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.287+0000 7f915e800640 1 -- 172.21.15.150:0/2425762157 >> v1:172.21.15.150:6789/0 conn(0x7f91601461c0 legacy=0x7f9160071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.287+0000 7f915e800640 1 -- 172.21.15.150:0/2425762157 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91600cd5b0 con 0x7f91600ce9e0 2024-04-15T15:24:35.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.288+0000 7f915e800640 1 --2- 172.21.15.150:0/2425762157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91600ce9e0 0x7f916006df20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f91600737f0 tx=0x7f915802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf6c86087e3752 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.288+0000 7f915d400640 1 -- 172.21.15.150:0/2425762157 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158039070 con 0x7f91600ce9e0 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.288+0000 7f915d400640 1 -- 172.21.15.150:0/2425762157 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915802cc80 con 0x7f91600ce9e0 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.288+0000 7f915d400640 1 -- 172.21.15.150:0/2425762157 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158034640 con 0x7f91600ce9e0 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.289+0000 7f915f200640 1 -- 172.21.15.150:0/2425762157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91600ce9e0 msgr2=0x7f916006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.289+0000 7f915f200640 1 --2- 172.21.15.150:0/2425762157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91600ce9e0 0x7f916006df20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f91600737f0 tx=0x7f915802bed0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.289+0000 7f915f200640 1 -- 172.21.15.150:0/2425762157 shutdown_connections 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.289+0000 7f915f200640 1 --2- 172.21.15.150:0/2425762157 >> v2:172.21.15.187:3300/0 conn(0x7f91600725f0 0x7f91600ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.289+0000 7f915f200640 1 --2- 172.21.15.150:0/2425762157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91600ce9e0 0x7f916006df20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.289+0000 7f915f200640 1 -- 172.21.15.150:0/2425762157 >> 172.21.15.150:0/2425762157 conn(0x7f916006b930 msgr2=0x7f916006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.289+0000 7f915f200640 1 -- 172.21.15.150:0/2425762157 shutdown_connections 2024-04-15T15:24:35.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.289+0000 7f915f200640 1 -- 172.21.15.150:0/2425762157 wait complete. 2024-04-15T15:24:35.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.291+0000 7f915f200640 1 Processor -- start 2024-04-15T15:24:35.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.291+0000 7f915f200640 1 -- start start 2024-04-15T15:24:35.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91600725f0 0x7f9160140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91600ce9e0 0x7f9160140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601461c0 0x7f91600aa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91600cee30 con 0x7f91601461c0 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91600cdb70 con 0x7f91600ce9e0 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f91600cefb0 con 0x7f91600725f0 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601461c0 0x7f91600aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601461c0 0x7f91600aa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36116/0 (socket says 172.21.15.150:36116) 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915e800640 1 -- 172.21.15.150:0/4121593479 learned_addr learned my addr 172.21.15.150:0/4121593479 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f9157e00640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91600ce9e0 0x7f9160140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915e800640 1 -- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91600725f0 msgr2=0x7f9160140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915e800640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91600725f0 0x7f9160140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915e800640 1 -- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91600ce9e0 msgr2=0x7f9160140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915e800640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91600ce9e0 0x7f9160140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.292+0000 7f915e800640 1 -- 172.21.15.150:0/4121593479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91601f56b0 con 0x7f91601461c0 2024-04-15T15:24:35.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.293+0000 7f915e800640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601461c0 0x7f91600aa110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9158009e80 tx=0x7f9158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:35.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.293+0000 7f9156000640 1 -- 172.21.15.150:0/4121593479 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158039050 con 0x7f91601461c0 2024-04-15T15:24:35.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.293+0000 7f9156000640 1 -- 172.21.15.150:0/4121593479 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9158045070 con 0x7f91601461c0 2024-04-15T15:24:35.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.293+0000 7f9156000640 1 -- 172.21.15.150:0/4121593479 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915803e830 con 0x7f91601461c0 2024-04-15T15:24:35.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.293+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91601405a0 con 0x7f91601461c0 2024-04-15T15:24:35.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.293+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91600b0310 con 0x7f91601461c0 2024-04-15T15:24:35.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.294+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91600755c0 con 0x7f91601461c0 2024-04-15T15:24:35.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.295+0000 7f9156000640 1 -- 172.21.15.150:0/4121593479 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f915803e9d0 con 0x7f91601461c0 2024-04-15T15:24:35.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.296+0000 7f9156000640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f913c06c8f0 0x7f913c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.296+0000 7f9156000640 1 -- 172.21.15.150:0/4121593479 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f91580ae870 con 0x7f91601461c0 2024-04-15T15:24:35.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.296+0000 7f915de00640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f913c06c8f0 0x7f913c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.298+0000 7f915de00640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f913c06c8f0 0x7f913c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9150000f60 tx=0x7f91500078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:35.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.301+0000 7f9156000640 1 -- 172.21.15.150:0/4121593479 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91601405a0 con 0x7f91601461c0 2024-04-15T15:24:35.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.503+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f91600cca50 con 0x7f91601461c0 2024-04-15T15:24:35.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.510+0000 7f9156000640 1 -- 172.21.15.150:0/4121593479 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f915807cc90 con 0x7f91601461c0 2024-04-15T15:24:35.512 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:35.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f913c06c8f0 msgr2=0x7f913c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f913c06c8f0 0x7f913c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9150000f60 tx=0x7f91500078f0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601461c0 msgr2=0x7f91600aa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601461c0 0x7f91600aa110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9158009e80 tx=0x7f9158004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 shutdown_connections 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601461c0 0x7f91600aa110 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91600ce9e0 0x7f9160140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f913c06c8f0 0x7f913c06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 --2- 172.21.15.150:0/4121593479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91600725f0 0x7f9160140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 >> 172.21.15.150:0/4121593479 conn(0x7f916006b930 msgr2=0x7f916006e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 shutdown_connections 2024-04-15T15:24:35.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.515+0000 7f915f200640 1 -- 172.21.15.150:0/4121593479 wait complete. 2024-04-15T15:24:35.556 INFO:tasks.cephfs_test_runner:Starting test: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:24:35.556 INFO:tasks.cephfs_test_runner:test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:24:35.557 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:35.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.754+0000 7f5d3a800640 1 Processor -- start 2024-04-15T15:24:35.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.754+0000 7f5d3a800640 1 -- start start 2024-04-15T15:24:35.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.755+0000 7f5d3a800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d341552e0 0x7f5d340afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.755+0000 7f5d3a800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5d3415c170 0x7f5d34072a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.755+0000 7f5d3a800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d340b48e0 con 0x7f5d341556c0 2024-04-15T15:24:35.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.755+0000 7f5d3a800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5d340b38d0 con 0x7f5d3415c170 2024-04-15T15:24:35.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.755+0000 7f5d3a800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d340b3b20 con 0x7f5d341552e0 2024-04-15T15:24:35.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.756+0000 7f5d2aa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5d3415c170 0x7f5d34072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.756+0000 7f5d33400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d341552e0 0x7f5d340afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.756+0000 7f5d2aa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5d3415c170 0x7f5d34072a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:35224/0 (socket says 172.21.15.150:35224) 2024-04-15T15:24:35.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.756+0000 7f5d2aa00640 1 -- 172.21.15.150:0/3184481670 learned_addr learned my addr 172.21.15.150:0/3184481670 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:35.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.756+0000 7f5d33400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d341552e0 0x7f5d340afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:47590/0 (socket says 172.21.15.150:47590) 2024-04-15T15:24:35.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.757+0000 7f5d33400640 1 -- 172.21.15.150:0/3184481670 >> v2:172.21.15.187:3300/0 conn(0x7f5d3415c170 msgr2=0x7f5d34072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:35.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.757+0000 7f5d33400640 1 --2- 172.21.15.150:0/3184481670 >> v2:172.21.15.187:3300/0 conn(0x7f5d3415c170 0x7f5d34072a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.757+0000 7f5d33400640 1 -- 172.21.15.150:0/3184481670 >> v1:172.21.15.187:6789/0 conn(0x7f5d341556c0 legacy=0x7f5d3415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.757+0000 7f5d33400640 1 -- 172.21.15.150:0/3184481670 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d340b3e00 con 0x7f5d341552e0 2024-04-15T15:24:35.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.759+0000 7f5d33400640 1 --2- 172.21.15.150:0/3184481670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d341552e0 0x7f5d340afb60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d240097a0 tx=0x7f5d2402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e49294af1491a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:35.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.759+0000 7f5d32a00640 1 -- 172.21.15.150:0/3184481670 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24035020 con 0x7f5d341552e0 2024-04-15T15:24:35.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.759+0000 7f5d32a00640 1 -- 172.21.15.150:0/3184481670 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d24031830 con 0x7f5d341552e0 2024-04-15T15:24:35.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.759+0000 7f5d32a00640 1 -- 172.21.15.150:0/3184481670 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24031c00 con 0x7f5d341552e0 2024-04-15T15:24:35.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.760+0000 7f5d3a800640 1 -- 172.21.15.150:0/3184481670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d341552e0 msgr2=0x7f5d340afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.760+0000 7f5d3a800640 1 --2- 172.21.15.150:0/3184481670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d341552e0 0x7f5d340afb60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d240097a0 tx=0x7f5d2402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.760+0000 7f5d3a800640 1 -- 172.21.15.150:0/3184481670 shutdown_connections 2024-04-15T15:24:35.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.760+0000 7f5d3a800640 1 --2- 172.21.15.150:0/3184481670 >> v2:172.21.15.187:3300/0 conn(0x7f5d3415c170 0x7f5d34072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.760+0000 7f5d3a800640 1 --2- 172.21.15.150:0/3184481670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d341552e0 0x7f5d340afb60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.760+0000 7f5d3a800640 1 -- 172.21.15.150:0/3184481670 >> 172.21.15.150:0/3184481670 conn(0x7f5d340b27b0 msgr2=0x7f5d340b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:35.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.760+0000 7f5d3a800640 1 -- 172.21.15.150:0/3184481670 shutdown_connections 2024-04-15T15:24:35.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.760+0000 7f5d3a800640 1 -- 172.21.15.150:0/3184481670 wait complete. 2024-04-15T15:24:35.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.762+0000 7f5d3a800640 1 Processor -- start 2024-04-15T15:24:35.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.762+0000 7f5d3a800640 1 -- start start 2024-04-15T15:24:35.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d3a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d341552e0 0x7f5d34146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d3a800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d341556c0 0x7f5d340721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d3a800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d3415c170 0x7f5d34072720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d3a800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5d340d0010 con 0x7f5d341552e0 2024-04-15T15:24:35.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d3a800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d340b43c0 con 0x7f5d341556c0 2024-04-15T15:24:35.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d3a800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d340d0190 con 0x7f5d3415c170 2024-04-15T15:24:35.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d33400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d341552e0 0x7f5d34146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d33400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d341552e0 0x7f5d34146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36120/0 (socket says 172.21.15.150:36120) 2024-04-15T15:24:35.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d33400640 1 -- 172.21.15.150:0/4236703774 learned_addr learned my addr 172.21.15.150:0/4236703774 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:35.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d2aa00640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d341556c0 0x7f5d340721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.763+0000 7f5d33e00640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d3415c170 0x7f5d34072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d33400640 1 -- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d3415c170 msgr2=0x7f5d34072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d33400640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d3415c170 0x7f5d34072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d33400640 1 -- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d341556c0 msgr2=0x7f5d340721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d33400640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d341556c0 0x7f5d340721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d33400640 1 -- 172.21.15.150:0/4236703774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d34151540 con 0x7f5d341552e0 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d33e00640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d3415c170 0x7f5d34072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d2aa00640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d341556c0 0x7f5d340721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d33400640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d341552e0 0x7f5d34146030 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d240098c0 tx=0x7f5d240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d2be00640 1 -- 172.21.15.150:0/4236703774 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24041070 con 0x7f5d341552e0 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d2be00640 1 -- 172.21.15.150:0/4236703774 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2403b920 con 0x7f5d341552e0 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d34156700 con 0x7f5d341552e0 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d2be00640 1 -- 172.21.15.150:0/4236703774 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24035040 con 0x7f5d341552e0 2024-04-15T15:24:35.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.764+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d34156da0 con 0x7f5d341552e0 2024-04-15T15:24:35.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.765+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d340b5c10 con 0x7f5d341552e0 2024-04-15T15:24:35.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.767+0000 7f5d2be00640 1 -- 172.21.15.150:0/4236703774 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d2404c060 con 0x7f5d341552e0 2024-04-15T15:24:35.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.767+0000 7f5d2be00640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5d1006c920 0x7f5d1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:35.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.768+0000 7f5d2be00640 1 -- 172.21.15.150:0/4236703774 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f5d240ad4a0 con 0x7f5d341552e0 2024-04-15T15:24:35.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.768+0000 7f5d2aa00640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5d1006c920 0x7f5d1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:35.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.769+0000 7f5d2aa00640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5d1006c920 0x7f5d1006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5d34070080 tx=0x7f5d1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:35.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.772+0000 7f5d2be00640 1 -- 172.21.15.150:0/4236703774 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d340b5c10 con 0x7f5d341552e0 2024-04-15T15:24:35.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.972+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d34068c80 con 0x7f5d341552e0 2024-04-15T15:24:35.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.974+0000 7f5d2be00640 1 -- 172.21.15.150:0/4236703774 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16997 (secure 0 0 0) 0x7f5d24031800 con 0x7f5d341552e0 2024-04-15T15:24:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:35.977 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":23,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:35.490857+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5d1006c920 msgr2=0x7f5d1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5d1006c920 0x7f5d1006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5d34070080 tx=0x7f5d1c008040 comp rx=0 tx=0).stop 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d341552e0 msgr2=0x7f5d34146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d341552e0 0x7f5d34146030 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d240098c0 tx=0x7f5d240046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 shutdown_connections 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5d1006c920 0x7f5d1006ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d3415c170 0x7f5d34072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d341556c0 0x7f5d340721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 --2- 172.21.15.150:0/4236703774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d341552e0 0x7f5d34146030 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 >> 172.21.15.150:0/4236703774 conn(0x7f5d340b27b0 msgr2=0x7f5d341532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:35.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 shutdown_connections 2024-04-15T15:24:35.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:35.977+0000 7f5d3a800640 1 -- 172.21.15.150:0/4236703774 wait complete. 2024-04-15T15:24:35.997 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:35.998 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:36.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.212+0000 7f4e31600640 1 Processor -- start 2024-04-15T15:24:36.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.212+0000 7f4e31600640 1 -- start start 2024-04-15T15:24:36.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.214+0000 7f4e31600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4e2c145170 0x7f4e2c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.214+0000 7f4e31600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c06f260 0x7f4e2c06f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.214+0000 7f4e31600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e2c06fbb0 con 0x7f4e2c0ce070 2024-04-15T15:24:36.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.214+0000 7f4e31600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f4e2c145550 con 0x7f4e2c145170 2024-04-15T15:24:36.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.214+0000 7f4e31600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2c0b4410 con 0x7f4e2c06f260 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.215+0000 7f4e2aa00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4e2c145170 0x7f4e2c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.215+0000 7f4e2aa00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4e2c145170 0x7f4e2c0cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36136/0 (socket says 172.21.15.150:36136) 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.215+0000 7f4e2aa00640 1 -- 172.21.15.150:0/971987599 learned_addr learned my addr 172.21.15.150:0/971987599 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.215+0000 7f4e2b400640 1 --2- 172.21.15.150:0/971987599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c06f260 0x7f4e2c06f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.215+0000 7f4e2aa00640 1 -- 172.21.15.150:0/971987599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c06f260 msgr2=0x7f4e2c06f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.215+0000 7f4e2aa00640 1 --2- 172.21.15.150:0/971987599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c06f260 0x7f4e2c06f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.215+0000 7f4e2aa00640 1 -- 172.21.15.150:0/971987599 >> v1:172.21.15.187:6789/0 conn(0x7f4e2c0ce070 legacy=0x7f4e2c0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.215+0000 7f4e2aa00640 1 -- 172.21.15.150:0/971987599 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c0b46f0 con 0x7f4e2c145170 2024-04-15T15:24:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.216+0000 7f4e2aa00640 1 --2- 172.21.15.150:0/971987599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c145170 0x7f4e2c0cdb30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009870 tx=0x7f4e1c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1f58f66b98a8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:36.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e29600640 1 -- 172.21.15.150:0/971987599 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c034070 con 0x7f4e2c145170 2024-04-15T15:24:36.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e29600640 1 -- 172.21.15.150:0/971987599 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1c030840 con 0x7f4e2c145170 2024-04-15T15:24:36.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e29600640 1 -- 172.21.15.150:0/971987599 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c030c10 con 0x7f4e2c145170 2024-04-15T15:24:36.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e31600640 1 -- 172.21.15.150:0/971987599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c145170 msgr2=0x7f4e2c0cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e31600640 1 --2- 172.21.15.150:0/971987599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c145170 0x7f4e2c0cdb30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009870 tx=0x7f4e1c02b980 comp rx=0 tx=0).stop 2024-04-15T15:24:36.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e31600640 1 -- 172.21.15.150:0/971987599 shutdown_connections 2024-04-15T15:24:36.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e31600640 1 --2- 172.21.15.150:0/971987599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c06f260 0x7f4e2c06f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e31600640 1 --2- 172.21.15.150:0/971987599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c145170 0x7f4e2c0cdb30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e31600640 1 -- 172.21.15.150:0/971987599 >> 172.21.15.150:0/971987599 conn(0x7f4e2c0b2460 msgr2=0x7f4e2c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:36.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e31600640 1 -- 172.21.15.150:0/971987599 shutdown_connections 2024-04-15T15:24:36.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.217+0000 7f4e31600640 1 -- 172.21.15.150:0/971987599 wait complete. 2024-04-15T15:24:36.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.219+0000 7f4e31600640 1 Processor -- start 2024-04-15T15:24:36.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.219+0000 7f4e31600640 1 -- start start 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.220+0000 7f4e31600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e2c06f260 0x7f4e2c137e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.220+0000 7f4e31600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c0ce070 0x7f4e2c0a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.220+0000 7f4e31600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c145170 0x7f4e2c1e13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.220+0000 7f4e31600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c0b4f90 con 0x7f4e2c0ce070 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.220+0000 7f4e31600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c0b4cb0 con 0x7f4e2c06f260 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.220+0000 7f4e31600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2c0b5bc0 con 0x7f4e2c145170 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c0ce070 0x7f4e2c0a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c0ce070 0x7f4e2c0a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36142/0 (socket says 172.21.15.150:36142) 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 -- 172.21.15.150:0/1241013469 learned_addr learned my addr 172.21.15.150:0/1241013469 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2aa00640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e2c06f260 0x7f4e2c137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2b400640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c145170 0x7f4e2c1e13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 -- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c145170 msgr2=0x7f4e2c1e13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c145170 0x7f4e2c1e13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 -- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e2c06f260 msgr2=0x7f4e2c137e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e2c06f260 0x7f4e2c137e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 -- 172.21.15.150:0/1241013469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c0aa7b0 con 0x7f4e2c0ce070 2024-04-15T15:24:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2b400640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c145170 0x7f4e2c1e13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:36.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.221+0000 7f4e2a000640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c0ce070 0x7f4e2c0a9eb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800c400 tx=0x7f4e1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.222+0000 7f4e22a00640 1 -- 172.21.15.150:0/1241013469 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e180052d0 con 0x7f4e2c0ce070 2024-04-15T15:24:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.222+0000 7f4e22a00640 1 -- 172.21.15.150:0/1241013469 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e18005490 con 0x7f4e2c0ce070 2024-04-15T15:24:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.222+0000 7f4e22a00640 1 -- 172.21.15.150:0/1241013469 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18007260 con 0x7f4e2c0ce070 2024-04-15T15:24:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.222+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c0aaa90 con 0x7f4e2c0ce070 2024-04-15T15:24:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.222+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c0cae00 con 0x7f4e2c0ce070 2024-04-15T15:24:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.223+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c068c80 con 0x7f4e2c0ce070 2024-04-15T15:24:36.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.224+0000 7f4e22a00640 1 -- 172.21.15.150:0/1241013469 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e18007400 con 0x7f4e2c0ce070 2024-04-15T15:24:36.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.225+0000 7f4e22a00640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.225+0000 7f4e22a00640 1 -- 172.21.15.150:0/1241013469 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f4e1808b130 con 0x7f4e2c0ce070 2024-04-15T15:24:36.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.226+0000 7f4e2aa00640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.227+0000 7f4e2aa00640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c036000 tx=0x7f4e1c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:36.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.229+0000 7f4e22a00640 1 -- 172.21.15.150:0/1241013469 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e2c06fbb0 con 0x7f4e2c0ce070 2024-04-15T15:24:36.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.428+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e2c15bb80 con 0x7f4e2c0ce070 2024-04-15T15:24:36.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.429+0000 7f4e22a00640 1 -- 172.21.15.150:0/1241013469 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f4e180594b0 con 0x7f4e2c0ce070 2024-04-15T15:24:36.434 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:36.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4e0c06c8f0 msgr2=0x7f4e0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c036000 tx=0x7f4e1c03b040 comp rx=0 tx=0).stop 2024-04-15T15:24:36.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c0ce070 msgr2=0x7f4e2c0a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c0ce070 0x7f4e2c0a9eb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800c400 tx=0x7f4e1800c8c0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 shutdown_connections 2024-04-15T15:24:36.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e2c145170 0x7f4e2c1e13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2c0ce070 0x7f4e2c0a9eb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 --2- 172.21.15.150:0/1241013469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e2c06f260 0x7f4e2c137e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 >> 172.21.15.150:0/1241013469 conn(0x7f4e2c0b2460 msgr2=0x7f4e2c0b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:36.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 shutdown_connections 2024-04-15T15:24:36.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.432+0000 7f4e31600640 1 -- 172.21.15.150:0/1241013469 wait complete. 2024-04-15T15:24:36.454 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:36.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.664+0000 7f7fb2200640 1 Processor -- start 2024-04-15T15:24:36.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.665+0000 7f7fb2200640 1 -- start start 2024-04-15T15:24:36.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.666+0000 7f7fb2200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7fac1461c0 0x7f7fac071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.666+0000 7f7fb2200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7fac0cebb0 0x7f7fac06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.666+0000 7f7fb2200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fac0cd2d0 con 0x7f7fac072420 2024-04-15T15:24:36.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.666+0000 7f7fb2200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f7fac0cf1d0 con 0x7f7fac0cebb0 2024-04-15T15:24:36.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.666+0000 7f7fb2200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7fac072880 con 0x7f7fac1461c0 2024-04-15T15:24:36.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.667+0000 7f7fb1800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7fac0cebb0 0x7f7fac06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.667+0000 7f7fb1800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7fac0cebb0 0x7f7fac06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36158/0 (socket says 172.21.15.150:36158) 2024-04-15T15:24:36.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.667+0000 7f7fb1800640 1 -- 172.21.15.150:0/713620768 learned_addr learned my addr 172.21.15.150:0/713620768 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:36.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.667+0000 7f7fb0e00640 1 --2- 172.21.15.150:0/713620768 >> v2:172.21.15.187:3300/0 conn(0x7f7fac1461c0 0x7f7fac071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.667+0000 7f7fb1800640 1 -- 172.21.15.150:0/713620768 >> v2:172.21.15.187:3300/0 conn(0x7f7fac1461c0 msgr2=0x7f7fac071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:36.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.667+0000 7f7fb1800640 1 --2- 172.21.15.150:0/713620768 >> v2:172.21.15.187:3300/0 conn(0x7f7fac1461c0 0x7f7fac071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.667+0000 7f7fb1800640 1 -- 172.21.15.150:0/713620768 >> v1:172.21.15.187:6789/0 conn(0x7f7fac072420 legacy=0x7f7fac0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:36.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.667+0000 7f7fb1800640 1 -- 172.21.15.150:0/713620768 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac0cd5b0 con 0x7f7fac0cebb0 2024-04-15T15:24:36.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.668+0000 7f7fb1800640 1 --2- 172.21.15.150:0/713620768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac0cebb0 0x7f7fac06df20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0d05c0 tx=0x7f7fa402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0ba2b64fa12d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:36.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.669+0000 7f7fab400640 1 -- 172.21.15.150:0/713620768 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa402ca80 con 0x7f7fac0cebb0 2024-04-15T15:24:36.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.669+0000 7f7fab400640 1 -- 172.21.15.150:0/713620768 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa402cc20 con 0x7f7fac0cebb0 2024-04-15T15:24:36.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.669+0000 7f7fab400640 1 -- 172.21.15.150:0/713620768 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40343f0 con 0x7f7fac0cebb0 2024-04-15T15:24:36.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.669+0000 7f7fb2200640 1 -- 172.21.15.150:0/713620768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac0cebb0 msgr2=0x7f7fac06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.669+0000 7f7fb2200640 1 --2- 172.21.15.150:0/713620768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac0cebb0 0x7f7fac06df20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0d05c0 tx=0x7f7fa402bee0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.669+0000 7f7fb2200640 1 -- 172.21.15.150:0/713620768 shutdown_connections 2024-04-15T15:24:36.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.670+0000 7f7fb2200640 1 --2- 172.21.15.150:0/713620768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac0cebb0 0x7f7fac06df20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.670+0000 7f7fb2200640 1 --2- 172.21.15.150:0/713620768 >> v2:172.21.15.187:3300/0 conn(0x7f7fac1461c0 0x7f7fac071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.670+0000 7f7fb2200640 1 -- 172.21.15.150:0/713620768 >> 172.21.15.150:0/713620768 conn(0x7f7fac06b930 msgr2=0x7f7fac06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:36.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.670+0000 7f7fb2200640 1 -- 172.21.15.150:0/713620768 shutdown_connections 2024-04-15T15:24:36.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.670+0000 7f7fb2200640 1 -- 172.21.15.150:0/713620768 wait complete. 2024-04-15T15:24:36.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.671+0000 7f7fb2200640 1 Processor -- start 2024-04-15T15:24:36.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.671+0000 7f7fb2200640 1 -- start start 2024-04-15T15:24:36.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.672+0000 7f7fb2200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fac072420 0x7f7fac1e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.672+0000 7f7fb2200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fac0cebb0 0x7f7fac1e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.672+0000 7f7fb2200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac1461c0 0x7f7fac144f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.672+0000 7f7fb2200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac0cdb70 con 0x7f7fac1461c0 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.672+0000 7f7fb2200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac0cd080 con 0x7f7fac072420 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.672+0000 7f7fb2200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac06e460 con 0x7f7fac0cebb0 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac1461c0 0x7f7fac144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac1461c0 0x7f7fac144f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36168/0 (socket says 172.21.15.150:36168) 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 -- 172.21.15.150:0/135057598 learned_addr learned my addr 172.21.15.150:0/135057598 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb0e00640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fac072420 0x7f7fac1e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fabe00640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fac0cebb0 0x7f7fac1e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 -- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fac0cebb0 msgr2=0x7f7fac1e2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fac0cebb0 0x7f7fac1e2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 -- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fac072420 msgr2=0x7f7fac1e4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fac072420 0x7f7fac1e4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 -- 172.21.15.150:0/135057598 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac1e3510 con 0x7f7fac1461c0 2024-04-15T15:24:36.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb0e00640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fac072420 0x7f7fac1e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:36.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.673+0000 7f7fb1800640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac1461c0 0x7f7fac144f70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009f40 tx=0x7f7fa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:36.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.674+0000 7f7fa8c00640 1 -- 172.21.15.150:0/135057598 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4043020 con 0x7f7fac1461c0 2024-04-15T15:24:36.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.674+0000 7f7fa8c00640 1 -- 172.21.15.150:0/135057598 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa403e450 con 0x7f7fac1461c0 2024-04-15T15:24:36.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.674+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac1e37f0 con 0x7f7fac1461c0 2024-04-15T15:24:36.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.674+0000 7f7fa8c00640 1 -- 172.21.15.150:0/135057598 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4009510 con 0x7f7fac1461c0 2024-04-15T15:24:36.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.674+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac1e4090 con 0x7f7fac1461c0 2024-04-15T15:24:36.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.675+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fac0755c0 con 0x7f7fac1461c0 2024-04-15T15:24:36.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.676+0000 7f7fa8c00640 1 -- 172.21.15.150:0/135057598 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fa4005d10 con 0x7f7fac1461c0 2024-04-15T15:24:36.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.677+0000 7f7fa8c00640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7f8c06c8f0 0x7f7f8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:36.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.677+0000 7f7fa8c00640 1 -- 172.21.15.150:0/135057598 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f7fa40ae7b0 con 0x7f7fac1461c0 2024-04-15T15:24:36.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.677+0000 7f7fb0e00640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7f8c06c8f0 0x7f7f8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:36.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.679+0000 7f7fb0e00640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7f8c06c8f0 0x7f7f8c06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0006fd0 tx=0x7f7fa0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:36.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.681+0000 7f7fa8c00640 1 -- 172.21.15.150:0/135057598 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fa407ca90 con 0x7f7fac1461c0 2024-04-15T15:24:36.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.880+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7fac1e39c0 con 0x7f7fac1461c0 2024-04-15T15:24:36.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.881+0000 7f7fa8c00640 1 -- 172.21.15.150:0/135057598 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f7fa407c390 con 0x7f7fac1461c0 2024-04-15T15:24:36.883 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:36.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7f8c06c8f0 msgr2=0x7f7f8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7f8c06c8f0 0x7f7f8c06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0006fd0 tx=0x7f7fa0008040 comp rx=0 tx=0).stop 2024-04-15T15:24:36.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac1461c0 msgr2=0x7f7fac144f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:36.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac1461c0 0x7f7fac144f70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009f40 tx=0x7f7fa4004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:36.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 shutdown_connections 2024-04-15T15:24:36.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fac1461c0 0x7f7fac144f70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7f8c06c8f0 0x7f7f8c06edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fac0cebb0 0x7f7fac1e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 --2- 172.21.15.150:0/135057598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fac072420 0x7f7fac1e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:36.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 >> 172.21.15.150:0/135057598 conn(0x7f7fac06b930 msgr2=0x7f7fac0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:36.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.884+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 shutdown_connections 2024-04-15T15:24:36.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:36.885+0000 7f7fb2200640 1 -- 172.21.15.150:0/135057598 wait complete. 2024-04-15T15:24:36.905 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:37.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.115+0000 7fa17ee00640 1 Processor -- start 2024-04-15T15:24:37.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.115+0000 7fa17ee00640 1 -- start start 2024-04-15T15:24:37.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.117+0000 7fa17ee00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa17815ec10 0x7fa17806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.117+0000 7fa17ee00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa1780b3930 0x7fa178160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.117+0000 7fa17ee00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17815f390 con 0x7fa1780b3dc0 2024-04-15T15:24:37.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.117+0000 7fa17ee00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa17815f670 con 0x7fa1780b3930 2024-04-15T15:24:37.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.117+0000 7fa17ee00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa17815f950 con 0x7fa17815ec10 2024-04-15T15:24:37.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.117+0000 7fa17e400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa1780b3930 0x7fa178160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.118+0000 7fa17e400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa1780b3930 0x7fa178160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35376/0 (socket says 172.21.15.150:35376) 2024-04-15T15:24:37.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.118+0000 7fa17e400640 1 -- 172.21.15.150:0/3333428885 learned_addr learned my addr 172.21.15.150:0/3333428885 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:37.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.118+0000 7fa17da00640 1 --2- 172.21.15.150:0/3333428885 >> v2:172.21.15.187:3300/0 conn(0x7fa17815ec10 0x7fa17806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.118+0000 7fa17e400640 1 -- 172.21.15.150:0/3333428885 >> v2:172.21.15.187:3300/0 conn(0x7fa17815ec10 msgr2=0x7fa17806bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:37.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.118+0000 7fa17e400640 1 --2- 172.21.15.150:0/3333428885 >> v2:172.21.15.187:3300/0 conn(0x7fa17815ec10 0x7fa17806bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.118+0000 7fa17e400640 1 -- 172.21.15.150:0/3333428885 >> v1:172.21.15.187:6789/0 conn(0x7fa1780b3dc0 legacy=0x7fa1780b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.118+0000 7fa17e400640 1 -- 172.21.15.150:0/3333428885 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17815fc30 con 0x7fa1780b3930 2024-04-15T15:24:37.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.119+0000 7fa17e400640 1 --2- 172.21.15.150:0/3333428885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa178160150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b4350 tx=0x7fa17002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540f3b4c17c16b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:37.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.119+0000 7fa177e00640 1 -- 172.21.15.150:0/3333428885 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17002cbb0 con 0x7fa1780b3930 2024-04-15T15:24:37.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.119+0000 7fa177e00640 1 -- 172.21.15.150:0/3333428885 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17002cd50 con 0x7fa1780b3930 2024-04-15T15:24:37.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.119+0000 7fa177e00640 1 -- 172.21.15.150:0/3333428885 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170034610 con 0x7fa1780b3930 2024-04-15T15:24:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.120+0000 7fa17ee00640 1 -- 172.21.15.150:0/3333428885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 msgr2=0x7fa178160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.120+0000 7fa17ee00640 1 --2- 172.21.15.150:0/3333428885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa178160150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b4350 tx=0x7fa17002c010 comp rx=0 tx=0).stop 2024-04-15T15:24:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.120+0000 7fa17ee00640 1 -- 172.21.15.150:0/3333428885 shutdown_connections 2024-04-15T15:24:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.120+0000 7fa17ee00640 1 --2- 172.21.15.150:0/3333428885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa178160150 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.120+0000 7fa17ee00640 1 --2- 172.21.15.150:0/3333428885 >> v2:172.21.15.187:3300/0 conn(0x7fa17815ec10 0x7fa17806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.120+0000 7fa17ee00640 1 -- 172.21.15.150:0/3333428885 >> 172.21.15.150:0/3333428885 conn(0x7fa17806ca40 msgr2=0x7fa17806ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.120+0000 7fa17ee00640 1 -- 172.21.15.150:0/3333428885 shutdown_connections 2024-04-15T15:24:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.120+0000 7fa17ee00640 1 -- 172.21.15.150:0/3333428885 wait complete. 2024-04-15T15:24:37.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.122+0000 7fa17ee00640 1 Processor -- start 2024-04-15T15:24:37.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.122+0000 7fa17ee00640 1 -- start start 2024-04-15T15:24:37.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.123+0000 7fa17ee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa1781f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.123+0000 7fa17ee00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1780b3dc0 0x7fa1781f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.123+0000 7fa17ee00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa17815ec10 0x7fa1781f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.123+0000 7fa17ee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa17806c5b0 con 0x7fa1780b3930 2024-04-15T15:24:37.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.123+0000 7fa17ee00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa17806c430 con 0x7fa1780b3dc0 2024-04-15T15:24:37.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.123+0000 7fa17ee00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa178160690 con 0x7fa17815ec10 2024-04-15T15:24:37.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa1781f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa1781f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35380/0 (socket says 172.21.15.150:35380) 2024-04-15T15:24:37.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 -- 172.21.15.150:0/854353604 learned_addr learned my addr 172.21.15.150:0/854353604 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:37.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17e400640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa17815ec10 0x7fa1781f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17d000640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1780b3dc0 0x7fa1781f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 -- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa17815ec10 msgr2=0x7fa1781f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa17815ec10 0x7fa1781f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 -- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1780b3dc0 msgr2=0x7fa1781f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1780b3dc0 0x7fa1781f80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 -- 172.21.15.150:0/854353604 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1780cd340 con 0x7fa1780b3930 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17e400640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa17815ec10 0x7fa1781f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17d000640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1780b3dc0 0x7fa1781f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.124+0000 7fa17da00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa1781f7b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00d5f0 tx=0x7fa16c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.125+0000 7fa175600640 1 -- 172.21.15.150:0/854353604 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0125f0 con 0x7fa1780b3930 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.125+0000 7fa175600640 1 -- 172.21.15.150:0/854353604 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16c019070 con 0x7fa1780b3930 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.125+0000 7fa175600640 1 -- 172.21.15.150:0/854353604 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0129d0 con 0x7fa1780b3930 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.125+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1780cd600 con 0x7fa1780b3930 2024-04-15T15:24:37.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.125+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa178157170 con 0x7fa1780b3930 2024-04-15T15:24:37.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.126+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1780755c0 con 0x7fa1780b3930 2024-04-15T15:24:37.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.127+0000 7fa175600640 1 -- 172.21.15.150:0/854353604 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa16c012b70 con 0x7fa1780b3930 2024-04-15T15:24:37.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.128+0000 7fa175600640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa15806c8f0 0x7fa15806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.128+0000 7fa175600640 1 -- 172.21.15.150:0/854353604 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fa16c08bff0 con 0x7fa1780b3930 2024-04-15T15:24:37.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.129+0000 7fa17d000640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa15806c8f0 0x7fa15806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.130+0000 7fa17d000640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa15806c8f0 0x7fa15806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1780cd2d0 tx=0x7fa164009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:37.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.132+0000 7fa175600640 1 -- 172.21.15.150:0/854353604 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa16c05a360 con 0x7fa1780b3930 2024-04-15T15:24:37.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.332+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa178069270 con 0x7fa1780b3930 2024-04-15T15:24:37.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.333+0000 7fa175600640 1 -- 172.21.15.150:0/854353604 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fa16c059c60 con 0x7fa1780b3930 2024-04-15T15:24:37.335 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa15806c8f0 msgr2=0x7fa15806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa15806c8f0 0x7fa15806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1780cd2d0 tx=0x7fa164009040 comp rx=0 tx=0).stop 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 msgr2=0x7fa1781f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa1781f7b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00d5f0 tx=0x7fa16c00dab0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 shutdown_connections 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa15806c8f0 0x7fa15806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa17815ec10 0x7fa1781f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1780b3dc0 0x7fa1781f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 --2- 172.21.15.150:0/854353604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1780b3930 0x7fa1781f7b80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 >> 172.21.15.150:0/854353604 conn(0x7fa17806ca40 msgr2=0x7fa178074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:37.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 shutdown_connections 2024-04-15T15:24:37.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.336+0000 7fa17ee00640 1 -- 172.21.15.150:0/854353604 wait complete. 2024-04-15T15:24:37.357 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked' 2024-04-15T15:24:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.562+0000 7f591be00640 1 Processor -- start 2024-04-15T15:24:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.562+0000 7f591be00640 1 -- start start 2024-04-15T15:24:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.564+0000 7f591be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f59141481f0 0x7f591415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.564+0000 7f591be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f59140affc0 0x7f59140721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.564+0000 7f591be00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59141485d0 con 0x7f59140b5560 2024-04-15T15:24:37.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.564+0000 7f591be00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f591415ffc0 con 0x7f59140affc0 2024-04-15T15:24:37.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.564+0000 7f591be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f59140aef20 con 0x7f59141481f0 2024-04-15T15:24:37.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.564+0000 7f5918a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f59140affc0 0x7f59140721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.564+0000 7f5918a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f59140affc0 0x7f59140721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35390/0 (socket says 172.21.15.150:35390) 2024-04-15T15:24:37.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.564+0000 7f5918a00640 1 -- 172.21.15.150:0/4199570933 learned_addr learned my addr 172.21.15.150:0/4199570933 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.565+0000 7f5919400640 1 --2- 172.21.15.150:0/4199570933 >> v2:172.21.15.187:3300/0 conn(0x7f59141481f0 0x7f591415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.565+0000 7f5913e00640 1 -- 172.21.15.150:0/4199570933 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 257956744 0 0) 0x7f59141485d0 con 0x7f59140b5560 2024-04-15T15:24:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.565+0000 7f5913e00640 1 -- 172.21.15.150:0/4199570933 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58fc003520 con 0x7f59140b5560 2024-04-15T15:24:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.565+0000 7f5918a00640 1 -- 172.21.15.150:0/4199570933 >> v2:172.21.15.187:3300/0 conn(0x7f59141481f0 msgr2=0x7f591415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.565+0000 7f5918a00640 1 --2- 172.21.15.150:0/4199570933 >> v2:172.21.15.187:3300/0 conn(0x7f59141481f0 0x7f591415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.565+0000 7f5918a00640 1 -- 172.21.15.150:0/4199570933 >> v1:172.21.15.150:6789/0 conn(0x7f59140b5560 legacy=0x7f59140b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.565+0000 7f5918a00640 1 -- 172.21.15.150:0/4199570933 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59140af200 con 0x7f59140affc0 2024-04-15T15:24:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.565+0000 7f5919400640 1 --2- 172.21.15.150:0/4199570933 >> v2:172.21.15.187:3300/0 conn(0x7f59141481f0 0x7f591415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.566+0000 7f5918a00640 1 --2- 172.21.15.150:0/4199570933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140affc0 0x7f59140721e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5914076c20 tx=0x7f590802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21f60760b710802 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.566+0000 7f5913e00640 1 -- 172.21.15.150:0/4199570933 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908034070 con 0x7f59140affc0 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.566+0000 7f5913e00640 1 -- 172.21.15.150:0/4199570933 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5908030860 con 0x7f59140affc0 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.566+0000 7f5913e00640 1 -- 172.21.15.150:0/4199570933 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908030c10 con 0x7f59140affc0 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.567+0000 7f591be00640 1 -- 172.21.15.150:0/4199570933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140affc0 msgr2=0x7f59140721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.567+0000 7f591be00640 1 --2- 172.21.15.150:0/4199570933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140affc0 0x7f59140721e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5914076c20 tx=0x7f590802b9a0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.567+0000 7f591be00640 1 -- 172.21.15.150:0/4199570933 shutdown_connections 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.567+0000 7f591be00640 1 --2- 172.21.15.150:0/4199570933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140affc0 0x7f59140721e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.567+0000 7f591be00640 1 --2- 172.21.15.150:0/4199570933 >> v2:172.21.15.187:3300/0 conn(0x7f59141481f0 0x7f591415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.567+0000 7f591be00640 1 -- 172.21.15.150:0/4199570933 >> 172.21.15.150:0/4199570933 conn(0x7f59140b27b0 msgr2=0x7f59140b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.567+0000 7f591be00640 1 -- 172.21.15.150:0/4199570933 shutdown_connections 2024-04-15T15:24:37.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.567+0000 7f591be00640 1 -- 172.21.15.150:0/4199570933 wait complete. 2024-04-15T15:24:37.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.569+0000 7f591be00640 1 Processor -- start 2024-04-15T15:24:37.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.569+0000 7f591be00640 1 -- start start 2024-04-15T15:24:37.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f591be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59140affc0 0x7f591415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f591be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140b5560 0x7f5914140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f591be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59141481f0 0x7f5914140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f591be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59140afaa0 con 0x7f59140b5560 2024-04-15T15:24:37.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f591be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59140af7c0 con 0x7f59141481f0 2024-04-15T15:24:37.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f591be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5914072720 con 0x7f59140affc0 2024-04-15T15:24:37.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f5918a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140b5560 0x7f5914140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f5918a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140b5560 0x7f5914140670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35404/0 (socket says 172.21.15.150:35404) 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.570+0000 7f5918a00640 1 -- 172.21.15.150:0/1646218452 learned_addr learned my addr 172.21.15.150:0/1646218452 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.571+0000 7f5919400640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59140affc0 0x7f591415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.571+0000 7f5919e00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59141481f0 0x7f5914140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.571+0000 7f5918a00640 1 -- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59140affc0 msgr2=0x7f591415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.571+0000 7f5918a00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59140affc0 0x7f591415e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.571+0000 7f5918a00640 1 -- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59141481f0 msgr2=0x7f5914140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.571+0000 7f5918a00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59141481f0 0x7f5914140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.571+0000 7f5918a00640 1 -- 172.21.15.150:0/1646218452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591406e2e0 con 0x7f59140b5560 2024-04-15T15:24:37.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.571+0000 7f5918a00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140b5560 0x7f5914140670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f590802b9e0 tx=0x7f59080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:37.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.572+0000 7f5911600640 1 -- 172.21.15.150:0/1646218452 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908047070 con 0x7f59140b5560 2024-04-15T15:24:37.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.572+0000 7f5911600640 1 -- 172.21.15.150:0/1646218452 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5908030660 con 0x7f59140b5560 2024-04-15T15:24:37.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.572+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591415e9d0 con 0x7f59140b5560 2024-04-15T15:24:37.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.572+0000 7f5911600640 1 -- 172.21.15.150:0/1646218452 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908034040 con 0x7f59140b5560 2024-04-15T15:24:37.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.572+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591406e830 con 0x7f59140b5560 2024-04-15T15:24:37.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.573+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5914068c80 con 0x7f59140b5560 2024-04-15T15:24:37.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.574+0000 7f5911600640 1 -- 172.21.15.150:0/1646218452 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f590804a070 con 0x7f59140b5560 2024-04-15T15:24:37.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.575+0000 7f5911600640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f58f406c920 0x7f58f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:37.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.575+0000 7f5911600640 1 -- 172.21.15.150:0/1646218452 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f59080ad820 con 0x7f59140b5560 2024-04-15T15:24:37.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.575+0000 7f5919400640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f58f406c920 0x7f58f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:37.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.577+0000 7f5919400640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f58f406c920 0x7f58f406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5900006fd0 tx=0x7f5900008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:37.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.579+0000 7f5911600640 1 -- 172.21.15.150:0/1646218452 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f590807bb10 con 0x7f59140b5560 2024-04-15T15:24:37.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:37.846+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]} v 0) -- 0x7f59140b2c30 con 0x7f59140b5560 2024-04-15T15:24:38.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.530+0000 7f5911600640 1 -- 172.21.15.150:0/1646218452 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]}]=0 v62) ==== 140+0+0 (secure 0 0 0) 0x7f590807b410 con 0x7f59140b5560 2024-04-15T15:24:38.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f58f406c920 msgr2=0x7f58f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f58f406c920 0x7f58f406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5900006fd0 tx=0x7f5900008040 comp rx=0 tx=0).stop 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140b5560 msgr2=0x7f5914140670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140b5560 0x7f5914140670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f590802b9e0 tx=0x7f59080046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 shutdown_connections 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59141481f0 0x7f5914140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59140b5560 0x7f5914140670 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f58f406c920 0x7f58f406ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 --2- 172.21.15.150:0/1646218452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59140affc0 0x7f591415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 >> 172.21.15.150:0/1646218452 conn(0x7f59140b27b0 msgr2=0x7f591413f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:38.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 shutdown_connections 2024-04-15T15:24:38.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.535+0000 7f591be00640 1 -- 172.21.15.150:0/1646218452 wait complete. 2024-04-15T15:24:38.561 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-15T15:24:38.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.774+0000 7f2028a00640 1 Processor -- start 2024-04-15T15:24:38.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.774+0000 7f2028a00640 1 -- start start 2024-04-15T15:24:38.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.776+0000 7f2028a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240afbc0 0x7f202415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:38.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.776+0000 7f2028a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f20240b36a0 0x7f20240b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:38.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.776+0000 7f2028a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f202415f310 con 0x7f202406ffd0 2024-04-15T15:24:38.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.776+0000 7f2028a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f20240cd460 con 0x7f20240b36a0 2024-04-15T15:24:38.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.776+0000 7f2028a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f202415e550 con 0x7f20240afbc0 2024-04-15T15:24:38.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.776+0000 7f2023e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f20240b36a0 0x7f20240b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:38.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.776+0000 7f2023e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f20240b36a0 0x7f20240b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35408/0 (socket says 172.21.15.150:35408) 2024-04-15T15:24:38.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.776+0000 7f2023e00640 1 -- 172.21.15.150:0/2745391132 learned_addr learned my addr 172.21.15.150:0/2745391132 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:38.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.777+0000 7f201be00640 1 --2- 172.21.15.150:0/2745391132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240afbc0 0x7f202415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:38.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.777+0000 7f2023e00640 1 -- 172.21.15.150:0/2745391132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240afbc0 msgr2=0x7f202415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:38.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.777+0000 7f2023e00640 1 --2- 172.21.15.150:0/2745391132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240afbc0 0x7f202415a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.777+0000 7f2023e00640 1 -- 172.21.15.150:0/2745391132 >> v1:172.21.15.187:6789/0 conn(0x7f202406ffd0 legacy=0x7f202406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:38.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.777+0000 7f2023e00640 1 -- 172.21.15.150:0/2745391132 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202415e830 con 0x7f20240b36a0 2024-04-15T15:24:38.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.778+0000 7f2023e00640 1 --2- 172.21.15.150:0/2745391132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f20240b3b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f201c009e50 tx=0x7f201c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576ba8935c639568 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:38.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.778+0000 7f2022a00640 1 -- 172.21.15.150:0/2745391132 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c02ca70 con 0x7f20240b36a0 2024-04-15T15:24:38.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.778+0000 7f2022a00640 1 -- 172.21.15.150:0/2745391132 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201c02cc10 con 0x7f20240b36a0 2024-04-15T15:24:38.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.778+0000 7f2022a00640 1 -- 172.21.15.150:0/2745391132 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c0343f0 con 0x7f20240b36a0 2024-04-15T15:24:38.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.779+0000 7f2028a00640 1 -- 172.21.15.150:0/2745391132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 msgr2=0x7f20240b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:38.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.779+0000 7f2028a00640 1 --2- 172.21.15.150:0/2745391132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f20240b3b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f201c009e50 tx=0x7f201c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.779+0000 7f2028a00640 1 -- 172.21.15.150:0/2745391132 shutdown_connections 2024-04-15T15:24:38.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.779+0000 7f2028a00640 1 --2- 172.21.15.150:0/2745391132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f20240b3b00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.779+0000 7f2028a00640 1 --2- 172.21.15.150:0/2745391132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20240afbc0 0x7f202415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.779+0000 7f2028a00640 1 -- 172.21.15.150:0/2745391132 >> 172.21.15.150:0/2745391132 conn(0x7f202406b930 msgr2=0x7f202406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:38.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.779+0000 7f2028a00640 1 -- 172.21.15.150:0/2745391132 shutdown_connections 2024-04-15T15:24:38.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.779+0000 7f2028a00640 1 -- 172.21.15.150:0/2745391132 wait complete. 2024-04-15T15:24:38.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.781+0000 7f2028a00640 1 Processor -- start 2024-04-15T15:24:38.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.781+0000 7f2028a00640 1 -- start start 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2028a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202406ffd0 0x7f202414a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2028a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240afbc0 0x7f2024154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2028a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f202414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2028a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20240b4040 con 0x7f20240b36a0 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2028a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f202415edf0 con 0x7f20240afbc0 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2028a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20240b41c0 con 0x7f202406ffd0 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2023e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f202414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2023e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f202414df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35410/0 (socket says 172.21.15.150:35410) 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.782+0000 7f2023e00640 1 -- 172.21.15.150:0/671335741 learned_addr learned my addr 172.21.15.150:0/671335741 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f201be00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240afbc0 0x7f2024154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:38.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2023e00640 1 -- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202406ffd0 msgr2=0x7f202414a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2023400640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202406ffd0 0x7f202414a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2023e00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202406ffd0 0x7f202414a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2023e00640 1 -- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240afbc0 msgr2=0x7f2024154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2023e00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240afbc0 0x7f2024154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2023e00640 1 -- 172.21.15.150:0/671335741 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2024152f60 con 0x7f20240b36a0 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f201be00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240afbc0 0x7f2024154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2023e00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f202414df10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f201c036040 tx=0x7f201c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f201b400640 1 -- 172.21.15.150:0/671335741 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c044020 con 0x7f20240b36a0 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f201b400640 1 -- 172.21.15.150:0/671335741 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201c040690 con 0x7f20240b36a0 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f201b400640 1 -- 172.21.15.150:0/671335741 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c040a80 con 0x7f20240b36a0 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2024153240 con 0x7f20240b36a0 2024-04-15T15:24:38.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.783+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2024153800 con 0x7f20240b36a0 2024-04-15T15:24:38.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.785+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20240755c0 con 0x7f20240b36a0 2024-04-15T15:24:38.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.785+0000 7f201b400640 1 -- 172.21.15.150:0/671335741 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f201c047020 con 0x7f20240b36a0 2024-04-15T15:24:38.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.786+0000 7f201b400640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f200006c920 0x7f200006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:38.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.786+0000 7f201b400640 1 -- 172.21.15.150:0/671335741 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f201c0ae520 con 0x7f20240b36a0 2024-04-15T15:24:38.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.787+0000 7f2023400640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f200006c920 0x7f200006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:38.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.788+0000 7f2023400640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f200006c920 0x7f200006ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2014003d90 tx=0x7f2014009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:38.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.790+0000 7f201b400640 1 -- 172.21.15.150:0/671335741 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f201c04e070 con 0x7f20240b36a0 2024-04-15T15:24:38.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.992+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f20240b1ba0 con 0x7f20240b36a0 2024-04-15T15:24:38.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:38.997+0000 7f201b400640 1 -- 172.21.15.150:0/671335741 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f201c07c8a0 con 0x7f20240b36a0 2024-04-15T15:24:39.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f200006c920 msgr2=0x7f200006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f200006c920 0x7f200006ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2014003d90 tx=0x7f2014009040 comp rx=0 tx=0).stop 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 msgr2=0x7f202414df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f202414df10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f201c036040 tx=0x7f201c004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 shutdown_connections 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20240b36a0 0x7f202414df10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20240afbc0 0x7f2024154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f200006c920 0x7f200006ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 --2- 172.21.15.150:0/671335741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202406ffd0 0x7f202414a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.000+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 >> 172.21.15.150:0/671335741 conn(0x7f202406b930 msgr2=0x7f20240ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.001+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 shutdown_connections 2024-04-15T15:24:39.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.001+0000 7f2028a00640 1 -- 172.21.15.150:0/671335741 wait complete. 2024-04-15T15:24:39.021 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-15T15:24:39.022 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:24:39.132 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:24:39.264 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-15T15:24:39.264 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:24:39.264 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:39.323 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:24:39.323 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:39.380 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:24:39.380 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:39.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:39.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.650+0000 7f1d6d000640 1 Processor -- start 2024-04-15T15:24:39.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.650+0000 7f1d6d000640 1 -- start start 2024-04-15T15:24:39.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.652+0000 7f1d6d000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1d6815e7c0 0x7f1d680b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:39.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.652+0000 7f1d6d000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d680b4be0 con 0x7f1d680cefc0 2024-04-15T15:24:39.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.652+0000 7f1d6d000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d680b4620 con 0x7f1d681461f0 2024-04-15T15:24:39.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.652+0000 7f1d6d000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1d680a9640 con 0x7f1d6815e7c0 2024-04-15T15:24:39.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.652+0000 7f1d5d400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1d680cefc0 0x7f1d680b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52888/0 (socket says 172.21.15.150:52888) 2024-04-15T15:24:39.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.652+0000 7f1d5d400640 1 -- 172.21.15.150:0/976043847 learned_addr learned my addr 172.21.15.150:0/976043847 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:39.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d66800640 1 --2- 172.21.15.150:0/976043847 >> v2:172.21.15.187:3300/0 conn(0x7f1d6815e7c0 0x7f1d680b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:39.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 661897480 0 0) 0x7f1d680b4be0 con 0x7f1d680cefc0 2024-04-15T15:24:39.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d4c003520 con 0x7f1d680cefc0 2024-04-15T15:24:39.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3889786675 0 0) 0x7f1d4c003520 con 0x7f1d680cefc0 2024-04-15T15:24:39.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 >> v2:172.21.15.187:3300/0 conn(0x7f1d6815e7c0 msgr2=0x7f1d680b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:39.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 --2- 172.21.15.150:0/976043847 >> v2:172.21.15.187:3300/0 conn(0x7f1d6815e7c0 0x7f1d680b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 >> v1:172.21.15.187:6789/0 conn(0x7f1d681461f0 legacy=0x7f1d680ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d680b4900 con 0x7f1d680cefc0 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f1d50001670 con 0x7f1d680cefc0 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 >> v1:172.21.15.150:6789/0 conn(0x7f1d680cefc0 legacy=0x7f1d680b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c003a00 0x7f1d4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d4c004800 0x7f1d4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d4c005210 0x7f1d4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c00c130 con 0x7f1d4c004800 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c00c2e0 con 0x7f1d4c005210 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.653+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c00c4e0 con 0x7f1d4c003a00 2024-04-15T15:24:39.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d5d400640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d4c004800 0x7f1d4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:39.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d65e00640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c003a00 0x7f1d4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:39.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d5d400640 1 -- 172.21.15.150:0/976043847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c003a00 msgr2=0x7f1d4c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:39.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d5d400640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c003a00 0x7f1d4c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d5d400640 1 -- 172.21.15.150:0/976043847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d4c005210 msgr2=0x7f1d4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:39.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d66800640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d4c005210 0x7f1d4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:39.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d5d400640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d4c005210 0x7f1d4c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d5d400640 1 -- 172.21.15.150:0/976043847 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c003520 con 0x7f1d4c004800 2024-04-15T15:24:39.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.654+0000 7f1d66800640 1 -- 172.21.15.150:0/976043847 reap_dead start 2024-04-15T15:24:39.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.655+0000 7f1d5d400640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d4c004800 0x7f1d4c004c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c0e0 tx=0x7f1d4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8737c4dea01a988 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:39.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.655+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50007c20 con 0x7f1d4c004800 2024-04-15T15:24:39.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d50030070 con 0x7f1d4c004800 2024-04-15T15:24:39.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d65400640 1 -- 172.21.15.150:0/976043847 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50005d70 con 0x7f1d4c004800 2024-04-15T15:24:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d6d000640 1 -- 172.21.15.150:0/976043847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d4c004800 msgr2=0x7f1d4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d6d000640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d4c004800 0x7f1d4c004c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c0e0 tx=0x7f1d4c005830 comp rx=0 tx=0).stop 2024-04-15T15:24:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d6d000640 1 -- 172.21.15.150:0/976043847 shutdown_connections 2024-04-15T15:24:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d6d000640 1 --2- 172.21.15.150:0/976043847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d4c004800 0x7f1d4c004c80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d6d000640 1 -- 172.21.15.150:0/976043847 >> 172.21.15.150:0/976043847 conn(0x7f1d680b27b0 msgr2=0x7f1d680b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d6d000640 1 -- 172.21.15.150:0/976043847 shutdown_connections 2024-04-15T15:24:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.656+0000 7f1d6d000640 1 -- 172.21.15.150:0/976043847 wait complete. 2024-04-15T15:24:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.658+0000 7f1d6d000640 1 Processor -- start 2024-04-15T15:24:39.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.658+0000 7f1d6d000640 1 -- start start 2024-04-15T15:24:39.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.659+0000 7f1d6d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c004800 0x7f1d681f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.659+0000 7f1d6d000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6806e320 0x7f1d6806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.659+0000 7f1d6d000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d681400f0 0x7f1d6815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.659+0000 7f1d6d000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681465d0 con 0x7f1d681400f0 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.659+0000 7f1d6d000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6815d290 con 0x7f1d6806e320 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.659+0000 7f1d6d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1d680ce8b0 con 0x7f1d4c004800 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.659+0000 7f1d66800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d681400f0 0x7f1d6815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.659+0000 7f1d66800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d681400f0 0x7f1d6815bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35432/0 (socket says 172.21.15.150:35432) 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d66800640 1 -- 172.21.15.150:0/2099603507 learned_addr learned my addr 172.21.15.150:0/2099603507 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d65e00640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c004800 0x7f1d681f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d5d400640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6806e320 0x7f1d6806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d66800640 1 -- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c004800 msgr2=0x7f1d681f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d66800640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c004800 0x7f1d681f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d66800640 1 -- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6806e320 msgr2=0x7f1d6806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d66800640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6806e320 0x7f1d6806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d66800640 1 -- 172.21.15.150:0/2099603507 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d680adef0 con 0x7f1d681400f0 2024-04-15T15:24:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.660+0000 7f1d66800640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d681400f0 0x7f1d6815bb60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d680b5d20 tx=0x7f1d6000e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.661+0000 7f1d5ea00640 1 -- 172.21.15.150:0/2099603507 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000c070 con 0x7f1d681400f0 2024-04-15T15:24:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.661+0000 7f1d5ea00640 1 -- 172.21.15.150:0/2099603507 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60018070 con 0x7f1d681400f0 2024-04-15T15:24:39.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.661+0000 7f1d5ea00640 1 -- 172.21.15.150:0/2099603507 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000f040 con 0x7f1d681400f0 2024-04-15T15:24:39.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.661+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d680ae190 con 0x7f1d681400f0 2024-04-15T15:24:39.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.661+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d680aea30 con 0x7f1d681400f0 2024-04-15T15:24:39.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.662+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d68068c80 con 0x7f1d681400f0 2024-04-15T15:24:39.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.663+0000 7f1d5ea00640 1 -- 172.21.15.150:0/2099603507 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d60003610 con 0x7f1d681400f0 2024-04-15T15:24:39.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.664+0000 7f1d5ea00640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1d4406c8f0 0x7f1d4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:39.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.664+0000 7f1d5ea00640 1 -- 172.21.15.150:0/2099603507 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f1d6008aea0 con 0x7f1d681400f0 2024-04-15T15:24:39.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.664+0000 7f1d65e00640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1d4406c8f0 0x7f1d4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:39.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.666+0000 7f1d65e00640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1d4406c8f0 0x7f1d4406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1d58007fd0 tx=0x7f1d58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:39.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.669+0000 7f1d5ea00640 1 -- 172.21.15.150:0/2099603507 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d60021020 con 0x7f1d681400f0 2024-04-15T15:24:39.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.871+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d680b0d30 con 0x7f1d681400f0 2024-04-15T15:24:39.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.873+0000 7f1d5ea00640 1 -- 172.21.15.150:0/2099603507 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16997 (secure 0 0 0) 0x7f1d60059210 con 0x7f1d681400f0 2024-04-15T15:24:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:39.875 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:36.500392+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:39.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.875+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1d4406c8f0 msgr2=0x7f1d4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1d4406c8f0 0x7f1d4406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1d58007fd0 tx=0x7f1d58009040 comp rx=0 tx=0).stop 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d681400f0 msgr2=0x7f1d6815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d681400f0 0x7f1d6815bb60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d680b5d20 tx=0x7f1d6000e830 comp rx=0 tx=0).stop 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 shutdown_connections 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d681400f0 0x7f1d6815bb60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6806e320 0x7f1d6806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1d4406c8f0 0x7f1d4406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 --2- 172.21.15.150:0/2099603507 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d4c004800 0x7f1d681f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 >> 172.21.15.150:0/2099603507 conn(0x7f1d680b27b0 msgr2=0x7f1d68145500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 shutdown_connections 2024-04-15T15:24:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:39.876+0000 7f1d6d000640 1 -- 172.21.15.150:0/2099603507 wait complete. 2024-04-15T15:24:39.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:39.898 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:40.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.109+0000 7fea0f800640 1 Processor -- start 2024-04-15T15:24:40.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.109+0000 7fea0f800640 1 -- start start 2024-04-15T15:24:40.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.110+0000 7fea0f800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fea0815a440 0x7fea0815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.110+0000 7fea0f800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea080725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.110+0000 7fea0f800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea0815d520 con 0x7fea0806fc80 2024-04-15T15:24:40.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.110+0000 7fea0f800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fea0806e280 con 0x7fea0815a440 2024-04-15T15:24:40.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.110+0000 7fea0f800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea0815d800 con 0x7fea08072150 2024-04-15T15:24:40.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea0ce00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fea0815a440 0x7fea0815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fe9ffe00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fea0806fc80 0x7fea080cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52896/0 (socket says 172.21.15.150:52896) 2024-04-15T15:24:40.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fe9ffe00640 1 -- 172.21.15.150:0/1261714238 learned_addr learned my addr 172.21.15.150:0/1261714238 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:40.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea0ce00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fea0815a440 0x7fea0815cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35440/0 (socket says 172.21.15.150:35440) 2024-04-15T15:24:40.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea0d800640 1 --2- 172.21.15.150:0/1261714238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea080725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea0ce00640 1 -- 172.21.15.150:0/1261714238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 msgr2=0x7fea080725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea0ce00640 1 --2- 172.21.15.150:0/1261714238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea080725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea0ce00640 1 -- 172.21.15.150:0/1261714238 >> v1:172.21.15.150:6789/0 conn(0x7fea0806fc80 legacy=0x7fea080cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea0ce00640 1 -- 172.21.15.150:0/1261714238 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0815dae0 con 0x7fea0815a440 2024-04-15T15:24:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea07e00640 1 -- 172.21.15.150:0/1261714238 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863721208 0 0) 0x7fea0815d520 con 0x7fea0806fc80 2024-04-15T15:24:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.111+0000 7fea0d800640 1 --2- 172.21.15.150:0/1261714238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea080725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.112+0000 7fea0ce00640 1 --2- 172.21.15.150:0/1261714238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0815a440 0x7fea0815cef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80097a0 tx=0x7fe9f802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef803924d8523b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea07e00640 1 -- 172.21.15.150:0/1261714238 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8035020 con 0x7fea0815a440 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea07e00640 1 -- 172.21.15.150:0/1261714238 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f8031810 con 0x7fea0815a440 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea07e00640 1 -- 172.21.15.150:0/1261714238 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8031be0 con 0x7fea0815a440 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea0f800640 1 -- 172.21.15.150:0/1261714238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0815a440 msgr2=0x7fea0815cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea0f800640 1 --2- 172.21.15.150:0/1261714238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0815a440 0x7fea0815cef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80097a0 tx=0x7fe9f802b8b0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea0f800640 1 -- 172.21.15.150:0/1261714238 shutdown_connections 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea0f800640 1 --2- 172.21.15.150:0/1261714238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea080725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea0f800640 1 --2- 172.21.15.150:0/1261714238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0815a440 0x7fea0815cef0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea0f800640 1 -- 172.21.15.150:0/1261714238 >> 172.21.15.150:0/1261714238 conn(0x7fea080b2460 msgr2=0x7fea080b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:40.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea0f800640 1 -- 172.21.15.150:0/1261714238 shutdown_connections 2024-04-15T15:24:40.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.113+0000 7fea0f800640 1 -- 172.21.15.150:0/1261714238 wait complete. 2024-04-15T15:24:40.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.115+0000 7fea0f800640 1 Processor -- start 2024-04-15T15:24:40.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.115+0000 7fea0f800640 1 -- start start 2024-04-15T15:24:40.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.116+0000 7fea0f800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0806fc80 0x7fea080b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.116+0000 7fea0f800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea08142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.116+0000 7fea0f800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea0815a440 0x7fea080abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.116+0000 7fea0f800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea0815ddc0 con 0x7fea0806fc80 2024-04-15T15:24:40.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.116+0000 7fea0f800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea0815a820 con 0x7fea0815a440 2024-04-15T15:24:40.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.116+0000 7fea0f800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea0806f510 con 0x7fea08072150 2024-04-15T15:24:40.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.116+0000 7fea0ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0806fc80 0x7fea080b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0806fc80 0x7fea080b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35450/0 (socket says 172.21.15.150:35450) 2024-04-15T15:24:40.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0ce00640 1 -- 172.21.15.150:0/2121651579 learned_addr learned my addr 172.21.15.150:0/2121651579 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fe9ffe00640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea08142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0d800640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea0815a440 0x7fea080abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0ce00640 1 -- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 msgr2=0x7fea08142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0ce00640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea08142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0ce00640 1 -- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea0815a440 msgr2=0x7fea080abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0ce00640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea0815a440 0x7fea080abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0ce00640 1 -- 172.21.15.150:0/2121651579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080adcc0 con 0x7fea0806fc80 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0d800640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea0815a440 0x7fea080abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.117+0000 7fea0ce00640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0806fc80 0x7fea080b3550 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80098c0 tx=0x7fe9f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.118+0000 7fea05600640 1 -- 172.21.15.150:0/2121651579 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8041070 con 0x7fea0806fc80 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.118+0000 7fea05600640 1 -- 172.21.15.150:0/2121651579 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f803b430 con 0x7fea0806fc80 2024-04-15T15:24:40.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.118+0000 7fea05600640 1 -- 172.21.15.150:0/2121651579 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8035040 con 0x7fea0806fc80 2024-04-15T15:24:40.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.118+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea080aa8a0 con 0x7fea0806fc80 2024-04-15T15:24:40.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.118+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea080ae210 con 0x7fea0806fc80 2024-04-15T15:24:40.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.119+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea08068c80 con 0x7fea0806fc80 2024-04-15T15:24:40.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.120+0000 7fea05600640 1 -- 172.21.15.150:0/2121651579 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9f804c060 con 0x7fea0806fc80 2024-04-15T15:24:40.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.121+0000 7fea05600640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.121+0000 7fea05600640 1 -- 172.21.15.150:0/2121651579 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fe9f80ad990 con 0x7fea0806fc80 2024-04-15T15:24:40.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.122+0000 7fe9ffe00640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.123+0000 7fe9ffe00640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e406c8f0 0x7fe9e406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea0806f3f0 tx=0x7fe9f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:40.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.128+0000 7fea05600640 1 -- 172.21.15.150:0/2121651579 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9f807bd00 con 0x7fea0806fc80 2024-04-15T15:24:40.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.328+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea080b29e0 con 0x7fea0806fc80 2024-04-15T15:24:40.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.329+0000 7fea05600640 1 -- 172.21.15.150:0/2121651579 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fe9f807b600 con 0x7fea0806fc80 2024-04-15T15:24:40.331 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:40.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e406c8f0 msgr2=0x7fe9e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e406c8f0 0x7fe9e406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea0806f3f0 tx=0x7fe9f0009040 comp rx=0 tx=0).stop 2024-04-15T15:24:40.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0806fc80 msgr2=0x7fea080b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0806fc80 0x7fea080b3550 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80098c0 tx=0x7fe9f80046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 shutdown_connections 2024-04-15T15:24:40.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea0815a440 0x7fea080abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea08072150 0x7fea08142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 --2- 172.21.15.150:0/2121651579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea0806fc80 0x7fea080b3550 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.331+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 >> 172.21.15.150:0/2121651579 conn(0x7fea080b2460 msgr2=0x7fea081449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:40.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.332+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 shutdown_connections 2024-04-15T15:24:40.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.332+0000 7fea0f800640 1 -- 172.21.15.150:0/2121651579 wait complete. 2024-04-15T15:24:40.353 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:40.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.557+0000 7fb957800640 1 Processor -- start 2024-04-15T15:24:40.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.557+0000 7fb957800640 1 -- start start 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.558+0000 7fb957800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95015ed60 0x7fb9500cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.558+0000 7fb957800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb9500ba220 0x7fb95006d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.558+0000 7fb957800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb9500ba660 0x7fb95006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.558+0000 7fb957800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fb9500d4180 con 0x7fb9500ba660 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.558+0000 7fb957800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb9500d35e0 con 0x7fb9500ba220 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.558+0000 7fb957800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb95015f140 con 0x7fb95015ed60 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb955800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb9500ba660 0x7fb95006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb955800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb9500ba660 0x7fb95006d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35462/0 (socket says 172.21.15.150:35462) 2024-04-15T15:24:40.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb955800640 1 -- 172.21.15.150:0/1261154330 learned_addr learned my addr 172.21.15.150:0/1261154330 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:40.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb947e00640 1 --2- 172.21.15.150:0/1261154330 >> v2:172.21.15.187:3300/0 conn(0x7fb9500ba220 0x7fb95006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb954e00640 1 --2- 172.21.15.150:0/1261154330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95015ed60 0x7fb9500cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb955800640 1 -- 172.21.15.150:0/1261154330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95015ed60 msgr2=0x7fb9500cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:40.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb955800640 1 --2- 172.21.15.150:0/1261154330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95015ed60 0x7fb9500cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb955800640 1 -- 172.21.15.150:0/1261154330 >> v2:172.21.15.187:3300/0 conn(0x7fb9500ba220 msgr2=0x7fb95006d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:40.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb955800640 1 --2- 172.21.15.150:0/1261154330 >> v2:172.21.15.187:3300/0 conn(0x7fb9500ba220 0x7fb95006d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb955800640 1 -- 172.21.15.150:0/1261154330 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9500d3850 con 0x7fb9500ba660 2024-04-15T15:24:40.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.559+0000 7fb947e00640 1 --2- 172.21.15.150:0/1261154330 >> v2:172.21.15.187:3300/0 conn(0x7fb9500ba220 0x7fb95006d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.560+0000 7fb955800640 1 --2- 172.21.15.150:0/1261154330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9500ba660 0x7fb95006d8a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb95008d290 tx=0x7fb94802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c166b0e975a4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb94fe00640 1 -- 172.21.15.150:0/1261154330 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94802cb00 con 0x7fb9500ba660 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb94fe00640 1 -- 172.21.15.150:0/1261154330 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94802cca0 con 0x7fb9500ba660 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb94fe00640 1 -- 172.21.15.150:0/1261154330 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948034610 con 0x7fb9500ba660 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 -- 172.21.15.150:0/1261154330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9500ba660 msgr2=0x7fb95006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 --2- 172.21.15.150:0/1261154330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9500ba660 0x7fb95006d8a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb95008d290 tx=0x7fb94802bf60 comp rx=0 tx=0).stop 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 -- 172.21.15.150:0/1261154330 shutdown_connections 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 --2- 172.21.15.150:0/1261154330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9500ba660 0x7fb95006d8a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 --2- 172.21.15.150:0/1261154330 >> v2:172.21.15.187:3300/0 conn(0x7fb9500ba220 0x7fb95006d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 --2- 172.21.15.150:0/1261154330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95015ed60 0x7fb9500cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 -- 172.21.15.150:0/1261154330 >> 172.21.15.150:0/1261154330 conn(0x7fb9500d4d70 msgr2=0x7fb9500d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:40.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 -- 172.21.15.150:0/1261154330 shutdown_connections 2024-04-15T15:24:40.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.561+0000 7fb957800640 1 -- 172.21.15.150:0/1261154330 wait complete. 2024-04-15T15:24:40.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.563+0000 7fb957800640 1 Processor -- start 2024-04-15T15:24:40.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.563+0000 7fb957800640 1 -- start start 2024-04-15T15:24:40.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb957800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9500ba220 0x7fb9500b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb957800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9500ba660 0x7fb9501f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb957800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95015ed60 0x7fb9500ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb957800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb9500d4610 con 0x7fb95015ed60 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb957800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb9500d3e10 con 0x7fb9500ba660 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb957800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb950161380 con 0x7fb9500ba220 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb955800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95015ed60 0x7fb9500ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb955800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95015ed60 0x7fb9500ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35478/0 (socket says 172.21.15.150:35478) 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.564+0000 7fb955800640 1 -- 172.21.15.150:0/1171780284 learned_addr learned my addr 172.21.15.150:0/1171780284 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.565+0000 7fb954e00640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9500ba220 0x7fb9500b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.565+0000 7fb947e00640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9500ba660 0x7fb9501f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.565+0000 7fb955800640 1 -- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9500ba220 msgr2=0x7fb9500b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.565+0000 7fb955800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9500ba220 0x7fb9500b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.565+0000 7fb955800640 1 -- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9500ba660 msgr2=0x7fb9501f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.565+0000 7fb955800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9500ba660 0x7fb9501f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.565+0000 7fb955800640 1 -- 172.21.15.150:0/1171780284 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb950160770 con 0x7fb95015ed60 2024-04-15T15:24:40.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.565+0000 7fb955800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95015ed60 0x7fb9500ab380 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb948036040 tx=0x7fb948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:40.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.566+0000 7fb94d600640 1 -- 172.21.15.150:0/1171780284 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948043020 con 0x7fb95015ed60 2024-04-15T15:24:40.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.566+0000 7fb94d600640 1 -- 172.21.15.150:0/1171780284 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94802ce50 con 0x7fb95015ed60 2024-04-15T15:24:40.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.566+0000 7fb94d600640 1 -- 172.21.15.150:0/1171780284 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94803e670 con 0x7fb95015ed60 2024-04-15T15:24:40.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.566+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb950160a50 con 0x7fb95015ed60 2024-04-15T15:24:40.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.566+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb950161010 con 0x7fb95015ed60 2024-04-15T15:24:40.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.567+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb950068c80 con 0x7fb95015ed60 2024-04-15T15:24:40.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.568+0000 7fb94d600640 1 -- 172.21.15.150:0/1171780284 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb948046020 con 0x7fb95015ed60 2024-04-15T15:24:40.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.570+0000 7fb94d600640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:40.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.570+0000 7fb94d600640 1 -- 172.21.15.150:0/1171780284 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fb9480ae340 con 0x7fb95015ed60 2024-04-15T15:24:40.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.570+0000 7fb954e00640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:40.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.571+0000 7fb954e00640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb92c06c8f0 0x7fb92c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb940006fd0 tx=0x7fb940008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:40.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.575+0000 7fb94d600640 1 -- 172.21.15.150:0/1171780284 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb94804d030 con 0x7fb95015ed60 2024-04-15T15:24:40.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.775+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb9500b7570 con 0x7fb95015ed60 2024-04-15T15:24:40.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.776+0000 7fb94d600640 1 -- 172.21.15.150:0/1171780284 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fb94807c6c0 con 0x7fb95015ed60 2024-04-15T15:24:40.778 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb92c06c8f0 msgr2=0x7fb92c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb92c06c8f0 0x7fb92c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb940006fd0 tx=0x7fb940008040 comp rx=0 tx=0).stop 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95015ed60 msgr2=0x7fb9500ab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95015ed60 0x7fb9500ab380 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb948036040 tx=0x7fb948004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 shutdown_connections 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95015ed60 0x7fb9500ab380 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9500ba660 0x7fb9501f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 --2- 172.21.15.150:0/1171780284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9500ba220 0x7fb9500b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 >> 172.21.15.150:0/1171780284 conn(0x7fb9500d4d70 msgr2=0x7fb9500d2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 shutdown_connections 2024-04-15T15:24:40.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:40.779+0000 7fb957800640 1 -- 172.21.15.150:0/1171780284 wait complete. 2024-04-15T15:24:40.800 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:41.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.011+0000 7f47d3a00640 1 Processor -- start 2024-04-15T15:24:41.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.011+0000 7f47d3a00640 1 -- start start 2024-04-15T15:24:41.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.013+0000 7f47d3a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f47cc162f90 0x7f47cc0ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.013+0000 7f47d3a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc0cec10 con 0x7f47cc0afbc0 2024-04-15T15:24:41.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.013+0000 7f47d3a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc06fe90 con 0x7f47cc072860 2024-04-15T15:24:41.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.013+0000 7f47d3a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f47cc0b46c0 con 0x7f47cc162f90 2024-04-15T15:24:41.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.013+0000 7f47cbe00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f47cc0afbc0 0x7f47cc072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52902/0 (socket says 172.21.15.150:52902) 2024-04-15T15:24:41.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.013+0000 7f47cbe00640 1 -- 172.21.15.150:0/3393728124 learned_addr learned my addr 172.21.15.150:0/3393728124 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:41.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.013+0000 7f47d1000640 1 --2- 172.21.15.150:0/3393728124 >> v2:172.21.15.187:3300/0 conn(0x7f47cc162f90 0x7f47cc0ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.013+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397076129 0 0) 0x7f47cc0cec10 con 0x7f47cc0afbc0 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47b4003520 con 0x7f47cc0afbc0 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4145803070 0 0) 0x7f47b4003520 con 0x7f47cc0afbc0 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 >> v2:172.21.15.187:3300/0 conn(0x7f47cc162f90 msgr2=0x7f47cc0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 --2- 172.21.15.150:0/3393728124 >> v2:172.21.15.187:3300/0 conn(0x7f47cc162f90 0x7f47cc0ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 >> v1:172.21.15.187:6789/0 conn(0x7f47cc072860 legacy=0x7f47cc166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc0b3ab0 con 0x7f47cc0afbc0 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f47b8001670 con 0x7f47cc0afbc0 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 >> v1:172.21.15.150:6789/0 conn(0x7f47cc0afbc0 legacy=0x7f47cc072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0b46c0 con 0x7f47b40039b0 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47b400c060 con 0x7f47b40051c0 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47b400c260 con 0x7f47b4004800 2024-04-15T15:24:41.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.014+0000 7f47d1000640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.015+0000 7f47d1a00640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.015+0000 7f47d1000640 1 -- 172.21.15.150:0/3393728124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b4004800 msgr2=0x7f47b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:41.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.015+0000 7f47d1000640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.015+0000 7f47d1000640 1 -- 172.21.15.150:0/3393728124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b40051c0 msgr2=0x7f47b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:41.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.015+0000 7f47d1000640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b40051c0 0x7f47b400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.015+0000 7f47d1a00640 1 -- 172.21.15.150:0/3393728124 reap_dead start 2024-04-15T15:24:41.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.015+0000 7f47d1000640 1 -- 172.21.15.150:0/3393728124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b4003520 con 0x7f47b40039b0 2024-04-15T15:24:41.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.016+0000 7f47d1000640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47b4003dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ac0 tx=0x7f47b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76e2239c15c0a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:41.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.016+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0039070 con 0x7f47b40039b0 2024-04-15T15:24:41.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.016+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c002c6b0 con 0x7f47b40039b0 2024-04-15T15:24:41.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.016+0000 7f47cb400640 1 -- 172.21.15.150:0/3393728124 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0009690 con 0x7f47b40039b0 2024-04-15T15:24:41.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.017+0000 7f47d3a00640 1 -- 172.21.15.150:0/3393728124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 msgr2=0x7f47b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.017+0000 7f47d3a00640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47b4003dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ac0 tx=0x7f47b4005830 comp rx=0 tx=0).stop 2024-04-15T15:24:41.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.017+0000 7f47d3a00640 1 -- 172.21.15.150:0/3393728124 shutdown_connections 2024-04-15T15:24:41.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.017+0000 7f47d3a00640 1 --2- 172.21.15.150:0/3393728124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47b4003dd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.017+0000 7f47d3a00640 1 -- 172.21.15.150:0/3393728124 >> 172.21.15.150:0/3393728124 conn(0x7f47cc0b2460 msgr2=0x7f47cc0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:41.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.017+0000 7f47d3a00640 1 -- 172.21.15.150:0/3393728124 shutdown_connections 2024-04-15T15:24:41.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.017+0000 7f47d3a00640 1 -- 172.21.15.150:0/3393728124 wait complete. 2024-04-15T15:24:41.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.019+0000 7f47d3a00640 1 Processor -- start 2024-04-15T15:24:41.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.019+0000 7f47d3a00640 1 -- start start 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d3a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47cc159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d3a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47cc147060 0x7f47cc161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d3a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47cc162060 0x7f47cc149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d3a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0b4350 con 0x7f47b40039b0 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d3a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0b4070 con 0x7f47cc162060 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d3a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47cc163370 con 0x7f47cc147060 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47cc159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47cc159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35492/0 (socket says 172.21.15.150:35492) 2024-04-15T15:24:41.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.020+0000 7f47d1000640 1 -- 172.21.15.150:0/2766234726 learned_addr learned my addr 172.21.15.150:0/2766234726 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47cbe00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47cc147060 0x7f47cc161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47d1a00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47cc162060 0x7f47cc149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47d1000640 1 -- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47cc147060 msgr2=0x7f47cc161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47d1000640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47cc147060 0x7f47cc161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47d1000640 1 -- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47cc162060 msgr2=0x7f47cc149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47d1000640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47cc162060 0x7f47cc149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47d1000640 1 -- 172.21.15.150:0/2766234726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc160a80 con 0x7f47b40039b0 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47cbe00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47cc147060 0x7f47cc161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47d1000640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47cc159f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ac0 tx=0x7f47c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47c8c00640 1 -- 172.21.15.150:0/2766234726 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0039050 con 0x7f47b40039b0 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47c8c00640 1 -- 172.21.15.150:0/2766234726 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c0045070 con 0x7f47b40039b0 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc160d60 con 0x7f47b40039b0 2024-04-15T15:24:41.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.021+0000 7f47c8c00640 1 -- 172.21.15.150:0/2766234726 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00044e0 con 0x7f47b40039b0 2024-04-15T15:24:41.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.022+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc161320 con 0x7f47b40039b0 2024-04-15T15:24:41.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.023+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc068c80 con 0x7f47b40039b0 2024-04-15T15:24:41.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.023+0000 7f47c8c00640 1 -- 172.21.15.150:0/2766234726 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47c0003a50 con 0x7f47b40039b0 2024-04-15T15:24:41.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.024+0000 7f47c8c00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.025+0000 7f47c8c00640 1 -- 172.21.15.150:0/2766234726 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f47c00ad670 con 0x7f47b40039b0 2024-04-15T15:24:41.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.025+0000 7f47cbe00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.026+0000 7f47cbe00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47ac06c8f0 0x7f47ac06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47b8007a30 tx=0x7f47b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:41.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.029+0000 7f47c8c00640 1 -- 172.21.15.150:0/2766234726 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47c004e020 con 0x7f47b40039b0 2024-04-15T15:24:41.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.229+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f47cc06f8d0 con 0x7f47b40039b0 2024-04-15T15:24:41.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.230+0000 7f47c8c00640 1 -- 172.21.15.150:0/2766234726 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f47c007ba80 con 0x7f47b40039b0 2024-04-15T15:24:41.232 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:41.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.233+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47ac06c8f0 msgr2=0x7f47ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.233+0000 7f47d3a00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47ac06c8f0 0x7f47ac06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47b8007a30 tx=0x7f47b8009040 comp rx=0 tx=0).stop 2024-04-15T15:24:41.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.233+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 msgr2=0x7f47cc159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.233+0000 7f47d3a00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47cc159f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ac0 tx=0x7f47c0004730 comp rx=0 tx=0).stop 2024-04-15T15:24:41.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.233+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 shutdown_connections 2024-04-15T15:24:41.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.233+0000 7f47d3a00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47cc162060 0x7f47cc149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.233+0000 7f47d3a00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.234+0000 7f47d3a00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47cc147060 0x7f47cc161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.234+0000 7f47d3a00640 1 --2- 172.21.15.150:0/2766234726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47b40039b0 0x7f47cc159f80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.234+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 >> 172.21.15.150:0/2766234726 conn(0x7f47cc0b2460 msgr2=0x7f47cc072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:41.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.234+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 shutdown_connections 2024-04-15T15:24:41.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.234+0000 7f47d3a00640 1 -- 172.21.15.150:0/2766234726 wait complete. 2024-04-15T15:24:41.253 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:24:41.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.464+0000 7fa29cc00640 1 Processor -- start 2024-04-15T15:24:41.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.464+0000 7fa29cc00640 1 -- start start 2024-04-15T15:24:41.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa29cc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa29813f6a0 0x7fa2980ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa29cc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa298072100 0x7fa298072580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa29cc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa29806e100 0x7fa29806e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa29cc00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa2980a95b0 con 0x7fa29813f6a0 2024-04-15T15:24:41.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa29cc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa29815d200 con 0x7fa29806e100 2024-04-15T15:24:41.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa29cc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa29815d4e0 con 0x7fa298072100 2024-04-15T15:24:41.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa295e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa29813f6a0 0x7fa2980ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa295e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa29813f6a0 0x7fa2980ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35504/0 (socket says 172.21.15.150:35504) 2024-04-15T15:24:41.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa295e00640 1 -- 172.21.15.150:0/2974963875 learned_addr learned my addr 172.21.15.150:0/2974963875 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:41.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.466+0000 7fa295400640 1 --2- 172.21.15.150:0/2974963875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa298072100 0x7fa298072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.467+0000 7fa296800640 1 --2- 172.21.15.150:0/2974963875 >> v2:172.21.15.187:3300/0 conn(0x7fa29806e100 0x7fa29806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.467+0000 7fa295e00640 1 -- 172.21.15.150:0/2974963875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa298072100 msgr2=0x7fa298072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:41.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.467+0000 7fa295e00640 1 --2- 172.21.15.150:0/2974963875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa298072100 0x7fa298072580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.467+0000 7fa295e00640 1 -- 172.21.15.150:0/2974963875 >> v2:172.21.15.187:3300/0 conn(0x7fa29806e100 msgr2=0x7fa29806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:41.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.467+0000 7fa295e00640 1 --2- 172.21.15.150:0/2974963875 >> v2:172.21.15.187:3300/0 conn(0x7fa29806e100 0x7fa29806e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.467+0000 7fa295e00640 1 -- 172.21.15.150:0/2974963875 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29815d7c0 con 0x7fa29813f6a0 2024-04-15T15:24:41.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.468+0000 7fa295e00640 1 --2- 172.21.15.150:0/2974963875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa29813f6a0 0x7fa2980ceda0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa2980ce190 tx=0x7fa28802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46f67ed2a661e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:41.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.469+0000 7fa294a00640 1 -- 172.21.15.150:0/2974963875 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288035020 con 0x7fa29813f6a0 2024-04-15T15:24:41.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.469+0000 7fa294a00640 1 -- 172.21.15.150:0/2974963875 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa288031870 con 0x7fa29813f6a0 2024-04-15T15:24:41.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.469+0000 7fa294a00640 1 -- 172.21.15.150:0/2974963875 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288031c40 con 0x7fa29813f6a0 2024-04-15T15:24:41.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 -- 172.21.15.150:0/2974963875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa29813f6a0 msgr2=0x7fa2980ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2974963875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa29813f6a0 0x7fa2980ceda0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa2980ce190 tx=0x7fa28802b910 comp rx=0 tx=0).stop 2024-04-15T15:24:41.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 -- 172.21.15.150:0/2974963875 shutdown_connections 2024-04-15T15:24:41.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2974963875 >> v2:172.21.15.187:3300/0 conn(0x7fa29806e100 0x7fa29806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2974963875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa298072100 0x7fa298072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2974963875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa29813f6a0 0x7fa2980ceda0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 -- 172.21.15.150:0/2974963875 >> 172.21.15.150:0/2974963875 conn(0x7fa2980b2460 msgr2=0x7fa2980b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:41.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 -- 172.21.15.150:0/2974963875 shutdown_connections 2024-04-15T15:24:41.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.470+0000 7fa29cc00640 1 -- 172.21.15.150:0/2974963875 wait complete. 2024-04-15T15:24:41.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.472+0000 7fa29cc00640 1 Processor -- start 2024-04-15T15:24:41.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.472+0000 7fa29cc00640 1 -- start start 2024-04-15T15:24:41.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa29cc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa29806e100 0x7fa298141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa29cc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa298072100 0x7fa298155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa29cc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa29813f6a0 0x7fa2981560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa29cc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa29815dd80 con 0x7fa298072100 2024-04-15T15:24:41.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa29cc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa29813fef0 con 0x7fa29806e100 2024-04-15T15:24:41.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa29cc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2980cf2e0 con 0x7fa29813f6a0 2024-04-15T15:24:41.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa295400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa298072100 0x7fa298155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa295400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa298072100 0x7fa298155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35506/0 (socket says 172.21.15.150:35506) 2024-04-15T15:24:41.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa295400640 1 -- 172.21.15.150:0/2096590470 learned_addr learned my addr 172.21.15.150:0/2096590470 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:41.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.473+0000 7fa295e00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa29806e100 0x7fa298141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa296800640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa29813f6a0 0x7fa2981560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa295400640 1 -- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa29813f6a0 msgr2=0x7fa2981560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa295400640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa29813f6a0 0x7fa2981560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa295400640 1 -- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa29806e100 msgr2=0x7fa298141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa295400640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa29806e100 0x7fa298141740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa295400640 1 -- 172.21.15.150:0/2096590470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa298156a20 con 0x7fa298072100 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa295400640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa298072100 0x7fa298155ba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e9b0 tx=0x7fa28400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa28e000640 1 -- 172.21.15.150:0/2096590470 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284017070 con 0x7fa298072100 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa28e000640 1 -- 172.21.15.150:0/2096590470 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa28400c9a0 con 0x7fa298072100 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa28e000640 1 -- 172.21.15.150:0/2096590470 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28400cd90 con 0x7fa298072100 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2980b0a90 con 0x7fa298072100 2024-04-15T15:24:41.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.474+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2980b1050 con 0x7fa298072100 2024-04-15T15:24:41.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.475+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa298068c80 con 0x7fa298072100 2024-04-15T15:24:41.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.476+0000 7fa28e000640 1 -- 172.21.15.150:0/2096590470 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa28401a070 con 0x7fa298072100 2024-04-15T15:24:41.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.477+0000 7fa28e000640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa27806c920 0x7fa27806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:41.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.477+0000 7fa28e000640 1 -- 172.21.15.150:0/2096590470 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fa28408bfa0 con 0x7fa298072100 2024-04-15T15:24:41.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.478+0000 7fa295e00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa27806c920 0x7fa27806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:41.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.479+0000 7fa295e00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa27806c920 0x7fa27806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa288036000 tx=0x7fa28803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:41.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.482+0000 7fa28e000640 1 -- 172.21.15.150:0/2096590470 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa298156a20 con 0x7fa298072100 2024-04-15T15:24:41.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.697+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2980b5850 con 0x7fa298072100 2024-04-15T15:24:41.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.698+0000 7fa28e000640 1 -- 172.21.15.150:0/2096590470 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fa28405a310 con 0x7fa298072100 2024-04-15T15:24:41.700 INFO:teuthology.orchestra.run.smithi150.stderr:listed 0 entries 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa27806c920 msgr2=0x7fa27806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa27806c920 0x7fa27806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa288036000 tx=0x7fa28803c040 comp rx=0 tx=0).stop 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa298072100 msgr2=0x7fa298155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa298072100 0x7fa298155ba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e9b0 tx=0x7fa28400ee70 comp rx=0 tx=0).stop 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 shutdown_connections 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa27806c920 0x7fa27806ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa29813f6a0 0x7fa2981560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa298072100 0x7fa298155ba0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 --2- 172.21.15.150:0/2096590470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa29806e100 0x7fa298141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 >> 172.21.15.150:0/2096590470 conn(0x7fa2980b2460 msgr2=0x7fa298135810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 shutdown_connections 2024-04-15T15:24:41.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:41.701+0000 7fa29cc00640 1 -- 172.21.15.150:0/2096590470 wait complete. 2024-04-15T15:24:41.722 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-15T15:24:41.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:41.722 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-15T15:24:41.821 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:24:41.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:41.821 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:24:41.879 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:24:41.880 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:41.939 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:24:41.939 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:41.992 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:24:41.992 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:42.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:42.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.266+0000 7fc96b400640 1 Processor -- start 2024-04-15T15:24:42.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.267+0000 7fc96b400640 1 -- start start 2024-04-15T15:24:42.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.268+0000 7fc96b400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc964070210 0x7fc9640b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.268+0000 7fc96b400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96406f940 con 0x7fc9640b5020 2024-04-15T15:24:42.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.268+0000 7fc96b400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96406e8a0 con 0x7fc96415b6d0 2024-04-15T15:24:42.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.268+0000 7fc96b400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fc96406eb80 con 0x7fc964070210 2024-04-15T15:24:42.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.268+0000 7fc95be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc964070210 0x7fc9640b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.268+0000 7fc95be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc964070210 0x7fc9640b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35522/0 (socket says 172.21.15.150:35522) 2024-04-15T15:24:42.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.268+0000 7fc95be00640 1 -- 172.21.15.150:0/2601103191 learned_addr learned my addr 172.21.15.150:0/2601103191 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:42.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.269+0000 7fc95be00640 1 -- 172.21.15.150:0/2601103191 >> v1:172.21.15.187:6789/0 conn(0x7fc96415b6d0 legacy=0x7fc9640cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:42.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.269+0000 7fc95be00640 1 -- 172.21.15.150:0/2601103191 >> v1:172.21.15.150:6789/0 conn(0x7fc9640b5020 legacy=0x7fc96415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.269+0000 7fc95be00640 1 -- 172.21.15.150:0/2601103191 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96406ee60 con 0x7fc964070210 2024-04-15T15:24:42.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.270+0000 7fc95be00640 1 --2- 172.21.15.150:0/2601103191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc964070210 0x7fc9640b4ae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc954009870 tx=0x7fc95402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecf78df72d5ad23 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:42.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.270+0000 7fc963e00640 1 -- 172.21.15.150:0/2601103191 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954034070 con 0x7fc964070210 2024-04-15T15:24:42.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.270+0000 7fc963e00640 1 -- 172.21.15.150:0/2601103191 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc954030860 con 0x7fc964070210 2024-04-15T15:24:42.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.270+0000 7fc963e00640 1 -- 172.21.15.150:0/2601103191 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954030c30 con 0x7fc964070210 2024-04-15T15:24:42.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.271+0000 7fc96b400640 1 -- 172.21.15.150:0/2601103191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc964070210 msgr2=0x7fc9640b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.271+0000 7fc96b400640 1 --2- 172.21.15.150:0/2601103191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc964070210 0x7fc9640b4ae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc954009870 tx=0x7fc95402b9a0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.271+0000 7fc96b400640 1 -- 172.21.15.150:0/2601103191 shutdown_connections 2024-04-15T15:24:42.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.271+0000 7fc96b400640 1 --2- 172.21.15.150:0/2601103191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc964070210 0x7fc9640b4ae0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.271+0000 7fc96b400640 1 -- 172.21.15.150:0/2601103191 >> 172.21.15.150:0/2601103191 conn(0x7fc9640b27b0 msgr2=0x7fc9640b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:42.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.271+0000 7fc96b400640 1 -- 172.21.15.150:0/2601103191 shutdown_connections 2024-04-15T15:24:42.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.271+0000 7fc96b400640 1 -- 172.21.15.150:0/2601103191 wait complete. 2024-04-15T15:24:42.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.273+0000 7fc96b400640 1 Processor -- start 2024-04-15T15:24:42.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.273+0000 7fc96b400640 1 -- start start 2024-04-15T15:24:42.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc96b400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc964070210 0x7fc9641f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc96b400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9640b5020 0x7fc964153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc96b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc96415b6d0 0x7fc964151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc96b400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc9640ce420 con 0x7fc96415b6d0 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc96b400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc96406f420 con 0x7fc9640b5020 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc96b400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc96415c1c0 con 0x7fc964070210 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc969400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc96415b6d0 0x7fc964151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc969400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc96415b6d0 0x7fc964151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35536/0 (socket says 172.21.15.150:35536) 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.274+0000 7fc969400640 1 -- 172.21.15.150:0/3025475462 learned_addr learned my addr 172.21.15.150:0/3025475462 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc968a00640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc964070210 0x7fc9641f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc95be00640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9640b5020 0x7fc964153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc969400640 1 -- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc964070210 msgr2=0x7fc9641f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc969400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc964070210 0x7fc9641f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc969400640 1 -- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9640b5020 msgr2=0x7fc964153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc969400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9640b5020 0x7fc964153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc969400640 1 -- 172.21.15.150:0/3025475462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc964153ff0 con 0x7fc96415b6d0 2024-04-15T15:24:42.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc95be00640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9640b5020 0x7fc964153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:42.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc969400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc96415b6d0 0x7fc964151a80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00c7a0 tx=0x7fc95c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:42.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc961600640 1 -- 172.21.15.150:0/3025475462 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c003b70 con 0x7fc96415b6d0 2024-04-15T15:24:42.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc961600640 1 -- 172.21.15.150:0/3025475462 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c003d30 con 0x7fc96415b6d0 2024-04-15T15:24:42.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc961600640 1 -- 172.21.15.150:0/3025475462 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00aa90 con 0x7fc96415b6d0 2024-04-15T15:24:42.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.275+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9641530b0 con 0x7fc96415b6d0 2024-04-15T15:24:42.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.276+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9641e1200 con 0x7fc96415b6d0 2024-04-15T15:24:42.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.277+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc964068c80 con 0x7fc96415b6d0 2024-04-15T15:24:42.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.278+0000 7fc961600640 1 -- 172.21.15.150:0/3025475462 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc95c00ac30 con 0x7fc96415b6d0 2024-04-15T15:24:42.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.279+0000 7fc961600640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc94006c8f0 0x7fc94006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.279+0000 7fc961600640 1 -- 172.21.15.150:0/3025475462 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fc95c08bb10 con 0x7fc96415b6d0 2024-04-15T15:24:42.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.280+0000 7fc968a00640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc94006c8f0 0x7fc94006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.281+0000 7fc968a00640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc94006c8f0 0x7fc94006edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc94c007fd0 tx=0x7fc94c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:42.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.284+0000 7fc961600640 1 -- 172.21.15.150:0/3025475462 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc964153ff0 con 0x7fc96415b6d0 2024-04-15T15:24:42.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.486+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9641527b0 con 0x7fc96415b6d0 2024-04-15T15:24:42.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.489+0000 7fc961600640 1 -- 172.21.15.150:0/3025475462 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16997 (secure 0 0 0) 0x7fc95c059f20 con 0x7fc96415b6d0 2024-04-15T15:24:42.491 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:42.491 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:36.500392+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.491+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc94006c8f0 msgr2=0x7fc94006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.491+0000 7fc96b400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc94006c8f0 0x7fc94006edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc94c007fd0 tx=0x7fc94c009040 comp rx=0 tx=0).stop 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.491+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc96415b6d0 msgr2=0x7fc964151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.491+0000 7fc96b400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc96415b6d0 0x7fc964151a80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00c7a0 tx=0x7fc95c00cc60 comp rx=0 tx=0).stop 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.492+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 shutdown_connections 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.492+0000 7fc96b400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc96415b6d0 0x7fc964151a80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.492+0000 7fc96b400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9640b5020 0x7fc964153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.492+0000 7fc96b400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc94006c8f0 0x7fc94006edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.492+0000 7fc96b400640 1 --2- 172.21.15.150:0/3025475462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc964070210 0x7fc9641f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.492+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 >> 172.21.15.150:0/3025475462 conn(0x7fc9640b27b0 msgr2=0x7fc9640b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.492+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 shutdown_connections 2024-04-15T15:24:42.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.492+0000 7fc96b400640 1 -- 172.21.15.150:0/3025475462 wait complete. 2024-04-15T15:24:42.514 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:42.514 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:42.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.721+0000 7f3e8ea00640 1 Processor -- start 2024-04-15T15:24:42.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.721+0000 7f3e8ea00640 1 -- start start 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e8ea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e880cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e8ea00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3e8806f260 0x7f3e8806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e8ea00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e880b4410 con 0x7f3e880ce070 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e8ea00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f3e88145550 con 0x7f3e8806f260 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e8ea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3e880b46f0 con 0x7f3e88145170 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e8ca00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3e8806f260 0x7f3e8806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e87400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3e880ce070 0x7f3e880b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52914/0 (socket says 172.21.15.150:52914) 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e8ca00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3e8806f260 0x7f3e8806f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35548/0 (socket says 172.21.15.150:35548) 2024-04-15T15:24:42.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e87400640 1 -- 172.21.15.150:0/578363265 learned_addr learned my addr 172.21.15.150:0/578363265 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:42.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.723+0000 7f3e87e00640 1 --2- 172.21.15.150:0/578363265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e880cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.724+0000 7f3e8ca00640 1 -- 172.21.15.150:0/578363265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 msgr2=0x7f3e880cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:42.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.724+0000 7f3e8ca00640 1 --2- 172.21.15.150:0/578363265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e880cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.724+0000 7f3e8ca00640 1 -- 172.21.15.150:0/578363265 >> v1:172.21.15.150:6789/0 conn(0x7f3e880ce070 legacy=0x7f3e880b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.724+0000 7f3e8ca00640 1 -- 172.21.15.150:0/578363265 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e880b49d0 con 0x7f3e8806f260 2024-04-15T15:24:42.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.724+0000 7f3e86a00640 1 -- 172.21.15.150:0/578363265 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1474242267 0 0) 0x7f3e880b4410 con 0x7f3e880ce070 2024-04-15T15:24:42.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.725+0000 7f3e8ca00640 1 --2- 172.21.15.150:0/578363265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e8806f6f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000bfa0 tx=0x7f3e8002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed3d8a03118dc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:42.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.725+0000 7f3e86a00640 1 -- 172.21.15.150:0/578363265 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8002ea40 con 0x7f3e8806f260 2024-04-15T15:24:42.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.725+0000 7f3e86a00640 1 -- 172.21.15.150:0/578363265 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8002ebe0 con 0x7f3e8806f260 2024-04-15T15:24:42.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.725+0000 7f3e86a00640 1 -- 172.21.15.150:0/578363265 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e80009540 con 0x7f3e8806f260 2024-04-15T15:24:42.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.726+0000 7f3e8ea00640 1 -- 172.21.15.150:0/578363265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 msgr2=0x7f3e8806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.726+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/578363265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e8806f6f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000bfa0 tx=0x7f3e8002e0a0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.726+0000 7f3e8ea00640 1 -- 172.21.15.150:0/578363265 shutdown_connections 2024-04-15T15:24:42.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.726+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/578363265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e8806f6f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.726+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/578363265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e880cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.726+0000 7f3e8ea00640 1 -- 172.21.15.150:0/578363265 >> 172.21.15.150:0/578363265 conn(0x7f3e880b2460 msgr2=0x7f3e880b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:42.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.726+0000 7f3e8ea00640 1 -- 172.21.15.150:0/578363265 shutdown_connections 2024-04-15T15:24:42.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.726+0000 7f3e8ea00640 1 -- 172.21.15.150:0/578363265 wait complete. 2024-04-15T15:24:42.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.728+0000 7f3e8ea00640 1 Processor -- start 2024-04-15T15:24:42.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.728+0000 7f3e8ea00640 1 -- start start 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e8ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e880b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e8ea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e880ce070 0x7f3e880aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e8ea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e8815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e8ea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3e880b5bc0 con 0x7f3e8806f260 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e8ea00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e880b4f90 con 0x7f3e880ce070 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e8ea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3e880b4030 con 0x7f3e88145170 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e87e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e880b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e87e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e880b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35562/0 (socket says 172.21.15.150:35562) 2024-04-15T15:24:42.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.729+0000 7f3e87e00640 1 -- 172.21.15.150:0/3618264576 learned_addr learned my addr 172.21.15.150:0/3618264576 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:42.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e8ca00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e8815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e87400640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e880ce070 0x7f3e880aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e87e00640 1 -- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 msgr2=0x7f3e8815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e87e00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e8815ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e87e00640 1 -- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e880ce070 msgr2=0x7f3e880aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e87e00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e880ce070 0x7f3e880aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e87e00640 1 -- 172.21.15.150:0/3618264576 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e880ac350 con 0x7f3e8806f260 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e8ca00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e8815ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e87400640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e880ce070 0x7f3e880aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e87e00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e880b0a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800d5f0 tx=0x7f3e7800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e7fe00640 1 -- 172.21.15.150:0/3618264576 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e780125f0 con 0x7f3e8806f260 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e7fe00640 1 -- 172.21.15.150:0/3618264576 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e78019070 con 0x7f3e8806f260 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e880ac5f0 con 0x7f3e8806f260 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.730+0000 7f3e7fe00640 1 -- 172.21.15.150:0/3618264576 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e780129d0 con 0x7f3e8806f260 2024-04-15T15:24:42.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.731+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e880acbb0 con 0x7f3e8806f260 2024-04-15T15:24:42.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.732+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e88068c60 con 0x7f3e8806f260 2024-04-15T15:24:42.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.733+0000 7f3e7fe00640 1 -- 172.21.15.150:0/3618264576 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e78012b70 con 0x7f3e8806f260 2024-04-15T15:24:42.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.733+0000 7f3e7fe00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3e6806c8f0 0x7f3e6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:42.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.734+0000 7f3e7fe00640 1 -- 172.21.15.150:0/3618264576 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f3e7808c0b0 con 0x7f3e8806f260 2024-04-15T15:24:42.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.734+0000 7f3e87400640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3e6806c8f0 0x7f3e6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:42.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.735+0000 7f3e87400640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3e6806c8f0 0x7f3e6806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e880abb50 tx=0x7f3e74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:42.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.738+0000 7f3e7fe00640 1 -- 172.21.15.150:0/3618264576 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e7805a390 con 0x7f3e8806f260 2024-04-15T15:24:42.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.938+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e8806d130 con 0x7f3e8806f260 2024-04-15T15:24:42.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.939+0000 7f3e7fe00640 1 -- 172.21.15.150:0/3618264576 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f3e78059c90 con 0x7f3e8806f260 2024-04-15T15:24:42.940 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3e6806c8f0 msgr2=0x7f3e6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3e6806c8f0 0x7f3e6806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e880abb50 tx=0x7f3e74009040 comp rx=0 tx=0).stop 2024-04-15T15:24:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 msgr2=0x7f3e880b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e880b0a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800d5f0 tx=0x7f3e7800dab0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 shutdown_connections 2024-04-15T15:24:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3e6806c8f0 0x7f3e6806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e88145170 0x7f3e8815ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e880ce070 0x7f3e880aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 --2- 172.21.15.150:0/3618264576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e8806f260 0x7f3e880b0a20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 >> 172.21.15.150:0/3618264576 conn(0x7f3e880b2460 msgr2=0x7f3e88144350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 shutdown_connections 2024-04-15T15:24:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:42.941+0000 7f3e8ea00640 1 -- 172.21.15.150:0/3618264576 wait complete. 2024-04-15T15:24:42.961 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:43.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.168+0000 7f0ad8200640 1 Processor -- start 2024-04-15T15:24:43.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.168+0000 7f0ad8200640 1 -- start start 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad8200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ad0162f90 0x7f0ad00ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad8200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ad0072860 0x7f0ad0166e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad8200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad00cec10 con 0x7f0ad00afbc0 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad8200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad00b46c0 con 0x7f0ad0072860 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad8200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad006fe90 con 0x7f0ad0162f90 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad6200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ad0072860 0x7f0ad0166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad6200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ad0072860 0x7f0ad0166e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35564/0 (socket says 172.21.15.150:35564) 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad6200640 1 -- 172.21.15.150:0/3349434060 learned_addr learned my addr 172.21.15.150:0/3349434060 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad4e00640 1 --1- 172.21.15.150:0/3349434060 >> v1:172.21.15.150:6789/0 conn(0x7f0ad00afbc0 0x7f0ad0072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52922/0 (socket says 172.21.15.150:52922) 2024-04-15T15:24:43.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.170+0000 7f0ad5800640 1 --2- 172.21.15.150:0/3349434060 >> v2:172.21.15.187:3300/0 conn(0x7f0ad0162f90 0x7f0ad00ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.171+0000 7f0ad6200640 1 -- 172.21.15.150:0/3349434060 >> v2:172.21.15.187:3300/0 conn(0x7f0ad0162f90 msgr2=0x7f0ad00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:43.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.171+0000 7f0ad6200640 1 --2- 172.21.15.150:0/3349434060 >> v2:172.21.15.187:3300/0 conn(0x7f0ad0162f90 0x7f0ad00ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.171+0000 7f0ad6200640 1 -- 172.21.15.150:0/3349434060 >> v1:172.21.15.150:6789/0 conn(0x7f0ad00afbc0 legacy=0x7f0ad0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.171+0000 7f0ad6200640 1 -- 172.21.15.150:0/3349434060 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad00b3ab0 con 0x7f0ad0072860 2024-04-15T15:24:43.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.171+0000 7f0acfe00640 1 -- 172.21.15.150:0/3349434060 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736740579 0 0) 0x7f0ad00cec10 con 0x7f0ad00afbc0 2024-04-15T15:24:43.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.172+0000 7f0ad6200640 1 --2- 172.21.15.150:0/3349434060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0072860 0x7f0ad0166e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0072e80 tx=0x7f0ac802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6200885e9a1c054 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:43.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.172+0000 7f0acfe00640 1 -- 172.21.15.150:0/3349434060 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac802cb20 con 0x7f0ad0072860 2024-04-15T15:24:43.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.172+0000 7f0acfe00640 1 -- 172.21.15.150:0/3349434060 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac802ccc0 con 0x7f0ad0072860 2024-04-15T15:24:43.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.172+0000 7f0acfe00640 1 -- 172.21.15.150:0/3349434060 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8034610 con 0x7f0ad0072860 2024-04-15T15:24:43.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.173+0000 7f0ad8200640 1 -- 172.21.15.150:0/3349434060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0072860 msgr2=0x7f0ad0166e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.173+0000 7f0ad8200640 1 --2- 172.21.15.150:0/3349434060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0072860 0x7f0ad0166e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0072e80 tx=0x7f0ac802bf80 comp rx=0 tx=0).stop 2024-04-15T15:24:43.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.173+0000 7f0ad8200640 1 -- 172.21.15.150:0/3349434060 shutdown_connections 2024-04-15T15:24:43.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.173+0000 7f0ad8200640 1 --2- 172.21.15.150:0/3349434060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0072860 0x7f0ad0166e30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.173+0000 7f0ad8200640 1 --2- 172.21.15.150:0/3349434060 >> v2:172.21.15.187:3300/0 conn(0x7f0ad0162f90 0x7f0ad00ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.173+0000 7f0ad8200640 1 -- 172.21.15.150:0/3349434060 >> 172.21.15.150:0/3349434060 conn(0x7f0ad00b2460 msgr2=0x7f0ad00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:43.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.173+0000 7f0ad8200640 1 -- 172.21.15.150:0/3349434060 shutdown_connections 2024-04-15T15:24:43.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.173+0000 7f0ad8200640 1 -- 172.21.15.150:0/3349434060 wait complete. 2024-04-15T15:24:43.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.175+0000 7f0ad8200640 1 Processor -- start 2024-04-15T15:24:43.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.175+0000 7f0ad8200640 1 -- start start 2024-04-15T15:24:43.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.175+0000 7f0ad8200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad0072860 0x7f0ad0154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad00afbc0 0x7f0ad01554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad8200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0162f90 0x7f0ad0159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad8200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad00b4350 con 0x7f0ad0162f90 2024-04-15T15:24:43.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad8200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad00b4070 con 0x7f0ad0072860 2024-04-15T15:24:43.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad0163370 con 0x7f0ad00afbc0 2024-04-15T15:24:43.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad6200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0162f90 0x7f0ad0159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad6200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0162f90 0x7f0ad0159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35576/0 (socket says 172.21.15.150:35576) 2024-04-15T15:24:43.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad6200640 1 -- 172.21.15.150:0/1596092366 learned_addr learned my addr 172.21.15.150:0/1596092366 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:43.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad5800640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad0072860 0x7f0ad0154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad6200640 1 -- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad00afbc0 msgr2=0x7f0ad01554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad6200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad00afbc0 0x7f0ad01554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad6200640 1 -- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad0072860 msgr2=0x7f0ad0154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad6200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad0072860 0x7f0ad0154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.176+0000 7f0ad6200640 1 -- 172.21.15.150:0/1596092366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad0155dd0 con 0x7f0ad0162f90 2024-04-15T15:24:43.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.177+0000 7f0ad6200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0162f90 0x7f0ad0159d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8036040 tx=0x7f0ac8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:43.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.177+0000 7f0acd600640 1 -- 172.21.15.150:0/1596092366 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8043020 con 0x7f0ad0162f90 2024-04-15T15:24:43.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.177+0000 7f0acd600640 1 -- 172.21.15.150:0/1596092366 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac803e450 con 0x7f0ad0162f90 2024-04-15T15:24:43.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.177+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad01560b0 con 0x7f0ad0162f90 2024-04-15T15:24:43.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.177+0000 7f0acd600640 1 -- 172.21.15.150:0/1596092366 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8009510 con 0x7f0ad0162f90 2024-04-15T15:24:43.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.177+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad0156670 con 0x7f0ad0162f90 2024-04-15T15:24:43.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.178+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad0068c80 con 0x7f0ad0162f90 2024-04-15T15:24:43.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.179+0000 7f0acd600640 1 -- 172.21.15.150:0/1596092366 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ac8046020 con 0x7f0ad0162f90 2024-04-15T15:24:43.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.180+0000 7f0acd600640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0ab006c8f0 0x7f0ab006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.180+0000 7f0acd600640 1 -- 172.21.15.150:0/1596092366 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f0ac80ae480 con 0x7f0ad0162f90 2024-04-15T15:24:43.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.181+0000 7f0ad5800640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0ab006c8f0 0x7f0ab006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.182+0000 7f0ad5800640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0ab006c8f0 0x7f0ab006edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4006fd0 tx=0x7f0ac4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:43.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.185+0000 7f0acd600640 1 -- 172.21.15.150:0/1596092366 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ac804d030 con 0x7f0ad0162f90 2024-04-15T15:24:43.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.378+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0ad00d0260 con 0x7f0ad0162f90 2024-04-15T15:24:43.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.379+0000 7f0acd600640 1 -- 172.21.15.150:0/1596092366 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f0ac807c800 con 0x7f0ad0162f90 2024-04-15T15:24:43.381 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:43.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0ab006c8f0 msgr2=0x7f0ab006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0ab006c8f0 0x7f0ab006edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4006fd0 tx=0x7f0ac4008040 comp rx=0 tx=0).stop 2024-04-15T15:24:43.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0162f90 msgr2=0x7f0ad0159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0162f90 0x7f0ad0159d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8036040 tx=0x7f0ac8004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 shutdown_connections 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad0162f90 0x7f0ad0159d00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0ab006c8f0 0x7f0ab006edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad00afbc0 0x7f0ad01554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 --2- 172.21.15.150:0/1596092366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad0072860 0x7f0ad0154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 >> 172.21.15.150:0/1596092366 conn(0x7f0ad00b2460 msgr2=0x7f0ad00cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 shutdown_connections 2024-04-15T15:24:43.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.382+0000 7f0ad8200640 1 -- 172.21.15.150:0/1596092366 wait complete. 2024-04-15T15:24:43.400 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:43.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.613+0000 7f595e400640 1 Processor -- start 2024-04-15T15:24:43.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.613+0000 7f595e400640 1 -- start start 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.614+0000 7f595e400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f59580725f0 0x7f59580ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.614+0000 7f595e400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f59580ce9e0 0x7f595806df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.614+0000 7f595e400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59580cd2d0 con 0x7f59581461c0 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.614+0000 7f595e400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f59580cf1d0 con 0x7f59580ce9e0 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.614+0000 7f595e400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f59580cd080 con 0x7f59580725f0 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f595da00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f59580ce9e0 0x7f595806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f595d000640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f59581461c0 0x7f5958071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52932/0 (socket says 172.21.15.150:52932) 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f595d000640 1 -- 172.21.15.150:0/3426400178 learned_addr learned my addr 172.21.15.150:0/3426400178 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f5957e00640 1 --2- 172.21.15.150:0/3426400178 >> v2:172.21.15.187:3300/0 conn(0x7f59580725f0 0x7f59580ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f595da00640 1 -- 172.21.15.150:0/3426400178 >> v2:172.21.15.187:3300/0 conn(0x7f59580725f0 msgr2=0x7f59580ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f595da00640 1 --2- 172.21.15.150:0/3426400178 >> v2:172.21.15.187:3300/0 conn(0x7f59580725f0 0x7f59580ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f595da00640 1 -- 172.21.15.150:0/3426400178 >> v1:172.21.15.150:6789/0 conn(0x7f59581461c0 legacy=0x7f5958071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f595da00640 1 -- 172.21.15.150:0/3426400178 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580cd5b0 con 0x7f59580ce9e0 2024-04-15T15:24:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.615+0000 7f5957400640 1 -- 172.21.15.150:0/3426400178 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4169612287 0 0) 0x7f59580cd2d0 con 0x7f59581461c0 2024-04-15T15:24:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.616+0000 7f595da00640 1 --2- 172.21.15.150:0/3426400178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580ce9e0 0x7f595806df20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f59580737f0 tx=0x7f595002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5c03bc52866002 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f5957400640 1 -- 172.21.15.150:0/3426400178 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950039070 con 0x7f59580ce9e0 2024-04-15T15:24:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f5957400640 1 -- 172.21.15.150:0/3426400178 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595002cc80 con 0x7f59580ce9e0 2024-04-15T15:24:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f5957400640 1 -- 172.21.15.150:0/3426400178 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950034660 con 0x7f59580ce9e0 2024-04-15T15:24:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f595e400640 1 -- 172.21.15.150:0/3426400178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580ce9e0 msgr2=0x7f595806df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f595e400640 1 --2- 172.21.15.150:0/3426400178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580ce9e0 0x7f595806df20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f59580737f0 tx=0x7f595002bed0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f595e400640 1 -- 172.21.15.150:0/3426400178 shutdown_connections 2024-04-15T15:24:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f595e400640 1 --2- 172.21.15.150:0/3426400178 >> v2:172.21.15.187:3300/0 conn(0x7f59580725f0 0x7f59580ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f595e400640 1 --2- 172.21.15.150:0/3426400178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580ce9e0 0x7f595806df20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f595e400640 1 -- 172.21.15.150:0/3426400178 >> 172.21.15.150:0/3426400178 conn(0x7f595806b930 msgr2=0x7f595806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f595e400640 1 -- 172.21.15.150:0/3426400178 shutdown_connections 2024-04-15T15:24:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.617+0000 7f595e400640 1 -- 172.21.15.150:0/3426400178 wait complete. 2024-04-15T15:24:43.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.619+0000 7f595e400640 1 Processor -- start 2024-04-15T15:24:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.619+0000 7f595e400640 1 -- start start 2024-04-15T15:24:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.620+0000 7f595e400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580725f0 0x7f595813eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.620+0000 7f595e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59580ce9e0 0x7f59581e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.620+0000 7f595e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59581461c0 0x7f59581e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.620+0000 7f595e400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59580cee30 con 0x7f59580725f0 2024-04-15T15:24:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.620+0000 7f595e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59580cdb70 con 0x7f59580ce9e0 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.620+0000 7f595e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59580cefb0 con 0x7f59581461c0 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.620+0000 7f595d000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580725f0 0x7f595813eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f595d000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580725f0 0x7f595813eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35598/0 (socket says 172.21.15.150:35598) 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f595d000640 1 -- 172.21.15.150:0/4151466580 learned_addr learned my addr 172.21.15.150:0/4151466580 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f5957e00640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59580ce9e0 0x7f59581e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f595d000640 1 -- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59581461c0 msgr2=0x7f59581e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f595d000640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59581461c0 0x7f59581e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f595d000640 1 -- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59580ce9e0 msgr2=0x7f59581e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f595d000640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59580ce9e0 0x7f59581e4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f595d000640 1 -- 172.21.15.150:0/4151466580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595813f0e0 con 0x7f59580725f0 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f5957e00640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59580ce9e0 0x7f59581e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.621+0000 7f595d000640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580725f0 0x7f595813eba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c430 tx=0x7f594c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.622+0000 7f5954c00640 1 -- 172.21.15.150:0/4151466580 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c0074e0 con 0x7f59580725f0 2024-04-15T15:24:43.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.622+0000 7f5954c00640 1 -- 172.21.15.150:0/4151466580 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c0047e0 con 0x7f59580725f0 2024-04-15T15:24:43.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.622+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59580b4ee0 con 0x7f59580725f0 2024-04-15T15:24:43.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.622+0000 7f5954c00640 1 -- 172.21.15.150:0/4151466580 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c004bd0 con 0x7f59580725f0 2024-04-15T15:24:43.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.622+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595812f180 con 0x7f59580725f0 2024-04-15T15:24:43.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.623+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59580b51c0 con 0x7f59580725f0 2024-04-15T15:24:43.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.624+0000 7f5954c00640 1 -- 172.21.15.150:0/4151466580 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f594c011070 con 0x7f59580725f0 2024-04-15T15:24:43.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.625+0000 7f5954c00640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f593806c8f0 0x7f593806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:43.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.625+0000 7f5954c00640 1 -- 172.21.15.150:0/4151466580 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f594c08b8c0 con 0x7f59580725f0 2024-04-15T15:24:43.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.625+0000 7f5957e00640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f593806c8f0 0x7f593806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:43.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.626+0000 7f5957e00640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f593806c8f0 0x7f593806edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59581e5af0 tx=0x7f5944008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:43.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.629+0000 7f5954c00640 1 -- 172.21.15.150:0/4151466580 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59580b51c0 con 0x7f59580725f0 2024-04-15T15:24:43.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.828+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f59580755c0 con 0x7f59580725f0 2024-04-15T15:24:43.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.828+0000 7f5954c00640 1 -- 172.21.15.150:0/4151466580 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f594c00e280 con 0x7f59580725f0 2024-04-15T15:24:43.830 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f593806c8f0 msgr2=0x7f593806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f593806c8f0 0x7f593806edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59581e5af0 tx=0x7f5944008040 comp rx=0 tx=0).stop 2024-04-15T15:24:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580725f0 msgr2=0x7f595813eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580725f0 0x7f595813eba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c430 tx=0x7f594c00c8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 shutdown_connections 2024-04-15T15:24:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f593806c8f0 0x7f593806edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59581461c0 0x7f59581e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59580ce9e0 0x7f59581e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 --2- 172.21.15.150:0/4151466580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59580725f0 0x7f595813eba0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:43.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 >> 172.21.15.150:0/4151466580 conn(0x7f595806b930 msgr2=0x7f595815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:43.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 shutdown_connections 2024-04-15T15:24:43.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:43.831+0000 7f595e400640 1 -- 172.21.15.150:0/4151466580 wait complete. 2024-04-15T15:24:43.852 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:24:44.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.061+0000 7f8d6d000640 1 Processor -- start 2024-04-15T15:24:44.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.061+0000 7f8d6d000640 1 -- start start 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.063+0000 7f8d6d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d680b0cb0 0x7f8d680ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.063+0000 7f8d6d000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8d680b59b0 0x7f8d680b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.063+0000 7f8d6d000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d680cd940 con 0x7f8d6815d770 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.063+0000 7f8d6d000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f8d680ce550 con 0x7f8d680b59b0 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.063+0000 7f8d6d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d680cdc20 con 0x7f8d680b0cb0 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.063+0000 7f8d67e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8d680b59b0 0x7f8d680b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.063+0000 7f8d66a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f8d6815d770 0x7f8d680cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52934/0 (socket says 172.21.15.150:52934) 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.063+0000 7f8d66a00640 1 -- 172.21.15.150:0/1037997656 learned_addr learned my addr 172.21.15.150:0/1037997656 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:44.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.064+0000 7f8d67400640 1 --2- 172.21.15.150:0/1037997656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d680b0cb0 0x7f8d680ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.064+0000 7f8d66000640 1 -- 172.21.15.150:0/1037997656 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034853434 0 0) 0x7f8d680cd940 con 0x7f8d6815d770 2024-04-15T15:24:44.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.064+0000 7f8d67e00640 1 -- 172.21.15.150:0/1037997656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d680b0cb0 msgr2=0x7f8d680ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:44.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.064+0000 7f8d67e00640 1 --2- 172.21.15.150:0/1037997656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d680b0cb0 0x7f8d680ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.064+0000 7f8d67e00640 1 -- 172.21.15.150:0/1037997656 >> v1:172.21.15.150:6789/0 conn(0x7f8d6815d770 legacy=0x7f8d680cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.064+0000 7f8d67e00640 1 -- 172.21.15.150:0/1037997656 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d680cdf00 con 0x7f8d680b59b0 2024-04-15T15:24:44.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.065+0000 7f8d67e00640 1 --2- 172.21.15.150:0/1037997656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b59b0 0x7f8d680b5e40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009d90 tx=0x7f8d6002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc3c934fe9109cd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:44.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.065+0000 7f8d66000640 1 -- 172.21.15.150:0/1037997656 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60039070 con 0x7f8d680b59b0 2024-04-15T15:24:44.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.065+0000 7f8d66000640 1 -- 172.21.15.150:0/1037997656 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6002cc50 con 0x7f8d680b59b0 2024-04-15T15:24:44.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.065+0000 7f8d66000640 1 -- 172.21.15.150:0/1037997656 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60034440 con 0x7f8d680b59b0 2024-04-15T15:24:44.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.066+0000 7f8d6d000640 1 -- 172.21.15.150:0/1037997656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b59b0 msgr2=0x7f8d680b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.066+0000 7f8d6d000640 1 --2- 172.21.15.150:0/1037997656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b59b0 0x7f8d680b5e40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009d90 tx=0x7f8d6002bea0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.066+0000 7f8d6d000640 1 -- 172.21.15.150:0/1037997656 shutdown_connections 2024-04-15T15:24:44.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.066+0000 7f8d6d000640 1 --2- 172.21.15.150:0/1037997656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b59b0 0x7f8d680b5e40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.066+0000 7f8d6d000640 1 --2- 172.21.15.150:0/1037997656 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d680b0cb0 0x7f8d680ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.066+0000 7f8d6d000640 1 -- 172.21.15.150:0/1037997656 >> 172.21.15.150:0/1037997656 conn(0x7f8d680b23a0 msgr2=0x7f8d680b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:44.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.066+0000 7f8d6d000640 1 -- 172.21.15.150:0/1037997656 shutdown_connections 2024-04-15T15:24:44.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.066+0000 7f8d6d000640 1 -- 172.21.15.150:0/1037997656 wait complete. 2024-04-15T15:24:44.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.068+0000 7f8d6d000640 1 Processor -- start 2024-04-15T15:24:44.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.068+0000 7f8d6d000640 1 -- start start 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d6d000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b0cb0 0x7f8d681f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d6d000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d680b59b0 0x7f8d68151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d6d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d6815d770 0x7f8d680aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d6d000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680cef20 con 0x7f8d680b0cb0 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d6d000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680b1090 con 0x7f8d680b59b0 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d6d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d681f9430 con 0x7f8d6815d770 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d67400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b0cb0 0x7f8d681f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d67400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b0cb0 0x7f8d681f8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35618/0 (socket says 172.21.15.150:35618) 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d67400640 1 -- 172.21.15.150:0/22187831 learned_addr learned my addr 172.21.15.150:0/22187831 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d67400640 1 -- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d6815d770 msgr2=0x7f8d680aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d67400640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d6815d770 0x7f8d680aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d67400640 1 -- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d680b59b0 msgr2=0x7f8d68151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d67400640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d680b59b0 0x7f8d68151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.069+0000 7f8d67400640 1 -- 172.21.15.150:0/22187831 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d68154770 con 0x7f8d680b0cb0 2024-04-15T15:24:44.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.070+0000 7f8d67400640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b0cb0 0x7f8d681f8ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800d620 tx=0x7f8d5800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:44.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.070+0000 7f8d5f400640 1 -- 172.21.15.150:0/22187831 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5800b620 con 0x7f8d680b0cb0 2024-04-15T15:24:44.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.070+0000 7f8d5f400640 1 -- 172.21.15.150:0/22187831 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d58019070 con 0x7f8d680b0cb0 2024-04-15T15:24:44.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.070+0000 7f8d5f400640 1 -- 172.21.15.150:0/22187831 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5800b9e0 con 0x7f8d680b0cb0 2024-04-15T15:24:44.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.070+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d68156160 con 0x7f8d680b0cb0 2024-04-15T15:24:44.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.070+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d68152530 con 0x7f8d680b0cb0 2024-04-15T15:24:44.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.071+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d680755c0 con 0x7f8d680b0cb0 2024-04-15T15:24:44.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.072+0000 7f8d5f400640 1 -- 172.21.15.150:0/22187831 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d5800bb80 con 0x7f8d680b0cb0 2024-04-15T15:24:44.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.073+0000 7f8d5f400640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d4406c920 0x7f8d4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.074+0000 7f8d5f400640 1 -- 172.21.15.150:0/22187831 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f8d5808bff0 con 0x7f8d680b0cb0 2024-04-15T15:24:44.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.074+0000 7f8d66a00640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d4406c920 0x7f8d4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.075+0000 7f8d66a00640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d4406c920 0x7f8d4406ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d50007fd0 tx=0x7f8d50007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:44.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.078+0000 7f8d5f400640 1 -- 172.21.15.150:0/22187831 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d5805a360 con 0x7f8d680b0cb0 2024-04-15T15:24:44.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.292+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8d68072f40 con 0x7f8d680b0cb0 2024-04-15T15:24:44.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.293+0000 7f8d5f400640 1 -- 172.21.15.150:0/22187831 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f8d58059c60 con 0x7f8d680b0cb0 2024-04-15T15:24:44.295 INFO:teuthology.orchestra.run.smithi150.stderr:listed 0 entries 2024-04-15T15:24:44.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d4406c920 msgr2=0x7f8d4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d4406c920 0x7f8d4406ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d50007fd0 tx=0x7f8d50007960 comp rx=0 tx=0).stop 2024-04-15T15:24:44.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b0cb0 msgr2=0x7f8d681f8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b0cb0 0x7f8d681f8ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800d620 tx=0x7f8d5800dae0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 shutdown_connections 2024-04-15T15:24:44.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d4406c920 0x7f8d4406ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d6815d770 0x7f8d680aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d680b59b0 0x7f8d68151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 --2- 172.21.15.150:0/22187831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d680b0cb0 0x7f8d681f8ef0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 >> 172.21.15.150:0/22187831 conn(0x7f8d680b23a0 msgr2=0x7f8d680b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:44.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.296+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 shutdown_connections 2024-04-15T15:24:44.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.297+0000 7f8d6d000640 1 -- 172.21.15.150:0/22187831 wait complete. 2024-04-15T15:24:44.316 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-15T15:24:44.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:44.316 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-15T15:24:44.423 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:24:44.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:44.423 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:24:44.483 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:24:44.483 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:44.537 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:24:44.538 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:44.591 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:24:44.591 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:44.638 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:44.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.898+0000 7fa799600640 1 Processor -- start 2024-04-15T15:24:44.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.898+0000 7fa799600640 1 -- start start 2024-04-15T15:24:44.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.899+0000 7fa799600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa7940cee00 0x7fa79406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.899+0000 7fa799600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa794071f50 0x7fa7940723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.899+0000 7fa799600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa79415cbe0 0x7fa79415d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.899+0000 7fa799600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa79406f4f0 con 0x7fa79415cbe0 2024-04-15T15:24:44.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.899+0000 7fa799600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa7940728f0 con 0x7fa7940cee00 2024-04-15T15:24:44.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.899+0000 7fa799600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa79415dbb0 con 0x7fa794071f50 2024-04-15T15:24:44.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa798c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa79415cbe0 0x7fa79415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa798c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa79415cbe0 0x7fa79415d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35630/0 (socket says 172.21.15.150:35630) 2024-04-15T15:24:44.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa798c00640 1 -- 172.21.15.150:0/2306775715 learned_addr learned my addr 172.21.15.150:0/2306775715 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:44.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa793e00640 1 --2- 172.21.15.150:0/2306775715 >> v2:172.21.15.187:3300/0 conn(0x7fa7940cee00 0x7fa79406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa78b400640 1 --2- 172.21.15.150:0/2306775715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa794071f50 0x7fa7940723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa798c00640 1 -- 172.21.15.150:0/2306775715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa794071f50 msgr2=0x7fa7940723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa798c00640 1 --2- 172.21.15.150:0/2306775715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa794071f50 0x7fa7940723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa798c00640 1 -- 172.21.15.150:0/2306775715 >> v2:172.21.15.187:3300/0 conn(0x7fa7940cee00 msgr2=0x7fa79406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa798c00640 1 --2- 172.21.15.150:0/2306775715 >> v2:172.21.15.187:3300/0 conn(0x7fa7940cee00 0x7fa79406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa798c00640 1 -- 172.21.15.150:0/2306775715 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79415d8d0 con 0x7fa79415cbe0 2024-04-15T15:24:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.900+0000 7fa78b400640 1 --2- 172.21.15.150:0/2306775715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa794071f50 0x7fa7940723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:44.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.902+0000 7fa798c00640 1 --2- 172.21.15.150:0/2306775715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa79415cbe0 0x7fa79415d070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009e50 tx=0x7fa78c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced125124e70937b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:44.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.902+0000 7fa793400640 1 -- 172.21.15.150:0/2306775715 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c02ca70 con 0x7fa79415cbe0 2024-04-15T15:24:44.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.902+0000 7fa793400640 1 -- 172.21.15.150:0/2306775715 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78c02cc10 con 0x7fa79415cbe0 2024-04-15T15:24:44.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.902+0000 7fa793400640 1 -- 172.21.15.150:0/2306775715 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0343f0 con 0x7fa79415cbe0 2024-04-15T15:24:44.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 -- 172.21.15.150:0/2306775715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa79415cbe0 msgr2=0x7fa79415d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 --2- 172.21.15.150:0/2306775715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa79415cbe0 0x7fa79415d070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009e50 tx=0x7fa78c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 -- 172.21.15.150:0/2306775715 shutdown_connections 2024-04-15T15:24:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 --2- 172.21.15.150:0/2306775715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa79415cbe0 0x7fa79415d070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 --2- 172.21.15.150:0/2306775715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa794071f50 0x7fa7940723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 --2- 172.21.15.150:0/2306775715 >> v2:172.21.15.187:3300/0 conn(0x7fa7940cee00 0x7fa79406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 -- 172.21.15.150:0/2306775715 >> 172.21.15.150:0/2306775715 conn(0x7fa79406b930 msgr2=0x7fa79406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 -- 172.21.15.150:0/2306775715 shutdown_connections 2024-04-15T15:24:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.903+0000 7fa799600640 1 -- 172.21.15.150:0/2306775715 wait complete. 2024-04-15T15:24:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.905+0000 7fa799600640 1 Processor -- start 2024-04-15T15:24:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.905+0000 7fa799600640 1 -- start start 2024-04-15T15:24:44.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.906+0000 7fa799600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa794071f50 0x7fa794152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.906+0000 7fa799600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7940cee00 0x7fa794143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.906+0000 7fa799600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa79415cbe0 0x7fa79415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.906+0000 7fa799600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa79406e3a0 con 0x7fa794071f50 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.906+0000 7fa799600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa79406e0c0 con 0x7fa7940cee00 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.906+0000 7fa799600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa79415d5b0 con 0x7fa79415cbe0 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa794071f50 0x7fa794152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa794071f50 0x7fa794152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35634/0 (socket says 172.21.15.150:35634) 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 -- 172.21.15.150:0/1831894816 learned_addr learned my addr 172.21.15.150:0/1831894816 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa78b400640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7940cee00 0x7fa794143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 -- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa79415cbe0 msgr2=0x7fa79415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa798c00640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa79415cbe0 0x7fa79415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa79415cbe0 0x7fa79415f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 -- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7940cee00 msgr2=0x7fa794143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7940cee00 0x7fa794143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:44.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 -- 172.21.15.150:0/1831894816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa794152ec0 con 0x7fa794071f50 2024-04-15T15:24:44.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.907+0000 7fa793e00640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa794071f50 0x7fa794152980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa78400d800 tx=0x7fa78400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:44.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.908+0000 7fa790c00640 1 -- 172.21.15.150:0/1831894816 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78400b6c0 con 0x7fa794071f50 2024-04-15T15:24:44.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.908+0000 7fa790c00640 1 -- 172.21.15.150:0/1831894816 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa784019070 con 0x7fa794071f50 2024-04-15T15:24:44.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.908+0000 7fa790c00640 1 -- 172.21.15.150:0/1831894816 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78400ba80 con 0x7fa794071f50 2024-04-15T15:24:44.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.908+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79415e740 con 0x7fa794071f50 2024-04-15T15:24:44.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.908+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79415efe0 con 0x7fa794071f50 2024-04-15T15:24:44.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.909+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79415eb30 con 0x7fa794071f50 2024-04-15T15:24:44.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.910+0000 7fa790c00640 1 -- 172.21.15.150:0/1831894816 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa78400bc20 con 0x7fa794071f50 2024-04-15T15:24:44.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.911+0000 7fa790c00640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa77006c8f0 0x7fa77006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:44.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.911+0000 7fa790c00640 1 -- 172.21.15.150:0/1831894816 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fa78408bfc0 con 0x7fa794071f50 2024-04-15T15:24:44.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.912+0000 7fa78b400640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa77006c8f0 0x7fa77006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:44.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.913+0000 7fa78b400640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa77006c8f0 0x7fa77006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00a950 tx=0x7fa77c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:44.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:44.916+0000 7fa790c00640 1 -- 172.21.15.150:0/1831894816 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa78405a3d0 con 0x7fa794071f50 2024-04-15T15:24:45.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.116+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7940755c0 con 0x7fa794071f50 2024-04-15T15:24:45.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.117+0000 7fa790c00640 1 -- 172.21.15.150:0/1831894816 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16997 (secure 0 0 0) 0x7fa784059cd0 con 0x7fa794071f50 2024-04-15T15:24:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:45.119 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:36.500392+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:45.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa77006c8f0 msgr2=0x7fa77006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa77006c8f0 0x7fa77006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00a950 tx=0x7fa77c008040 comp rx=0 tx=0).stop 2024-04-15T15:24:45.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa794071f50 msgr2=0x7fa794152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa794071f50 0x7fa794152980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa78400d800 tx=0x7fa78400dcc0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 shutdown_connections 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa77006c8f0 0x7fa77006edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa79415cbe0 0x7fa79415f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7940cee00 0x7fa794143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 --2- 172.21.15.150:0/1831894816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa794071f50 0x7fa794152980 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 >> 172.21.15.150:0/1831894816 conn(0x7fa79406b930 msgr2=0x7fa7940af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 shutdown_connections 2024-04-15T15:24:45.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.120+0000 7fa799600640 1 -- 172.21.15.150:0/1831894816 wait complete. 2024-04-15T15:24:45.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:45.142 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:45.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.342+0000 7f384e800640 1 Processor -- start 2024-04-15T15:24:45.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.342+0000 7f384e800640 1 -- start start 2024-04-15T15:24:45.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384e800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f38480d4e30 0x7f38480d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38480cede0 0x7f38480cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384e800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f384815ec60 con 0x7f384815e880 2024-04-15T15:24:45.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384e800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f38480cfaf0 con 0x7f38480d4e30 2024-04-15T15:24:45.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384e800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f38480d2930 con 0x7f38480cede0 2024-04-15T15:24:45.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384d400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f384815e880 0x7f38480d4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52946/0 (socket says 172.21.15.150:52946) 2024-04-15T15:24:45.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384d400640 1 -- 172.21.15.150:0/26125074 learned_addr learned my addr 172.21.15.150:0/26125074 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:45.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384ca00640 1 --2- 172.21.15.150:0/26125074 >> v2:172.21.15.187:3300/0 conn(0x7f38480d4e30 0x7f38480d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f384de00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38480cede0 0x7f38480cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.344+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795278991 0 0) 0x7f384815ec60 con 0x7f384815e880 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3830003520 con 0x7f384815e880 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1055906348 0 0) 0x7f3830003520 con 0x7f384815e880 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38480cede0 msgr2=0x7f38480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38480cede0 0x7f38480cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 >> v2:172.21.15.187:3300/0 conn(0x7f38480d4e30 msgr2=0x7f38480d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 --2- 172.21.15.150:0/26125074 >> v2:172.21.15.187:3300/0 conn(0x7f38480d4e30 0x7f38480d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38480d2c10 con 0x7f384815e880 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f383c001670 con 0x7f384815e880 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 >> v1:172.21.15.150:6789/0 conn(0x7f384815e880 legacy=0x7f38480d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38480cede0 0x7f38300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3830003f30 0x7f3830004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3830004d40 0x7f38300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38480cfaf0 con 0x7f3830003f30 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38480d2930 con 0x7f3830004d40 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f383000b9f0 con 0x7f38480cede0 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.345+0000 7f384ca00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3830003f30 0x7f3830004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.346+0000 7f384d400640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38480cede0 0x7f38300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.346+0000 7f384de00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3830004d40 0x7f38300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.346+0000 7f384ca00640 1 -- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38480cede0 msgr2=0x7f38300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.346+0000 7f384ca00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38480cede0 0x7f38300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.346+0000 7f384ca00640 1 -- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3830004d40 msgr2=0x7f38300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.346+0000 7f384ca00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3830004d40 0x7f38300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.346+0000 7f384ca00640 1 -- 172.21.15.150:0/26125074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3830003520 con 0x7f3830003f30 2024-04-15T15:24:45.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.346+0000 7f384de00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3830004d40 0x7f38300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:45.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.347+0000 7f384ca00640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3830003f30 0x7f3830004800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3834009ac0 tx=0x7f3830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caba3c6a66c356b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:45.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.347+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834039070 con 0x7f3830003f30 2024-04-15T15:24:45.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.347+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383402c710 con 0x7f3830003f30 2024-04-15T15:24:45.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.347+0000 7f3847e00640 1 -- 172.21.15.150:0/26125074 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834009690 con 0x7f3830003f30 2024-04-15T15:24:45.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.348+0000 7f384e800640 1 -- 172.21.15.150:0/26125074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3830003f30 msgr2=0x7f3830004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.348+0000 7f384e800640 1 --2- 172.21.15.150:0/26125074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3830003f30 0x7f3830004800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3834009ac0 tx=0x7f3830005830 comp rx=0 tx=0).stop 2024-04-15T15:24:45.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.348+0000 7f384de00640 1 -- 172.21.15.150:0/26125074 reap_dead start 2024-04-15T15:24:45.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.348+0000 7f384e800640 1 -- 172.21.15.150:0/26125074 shutdown_connections 2024-04-15T15:24:45.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.348+0000 7f384e800640 1 -- 172.21.15.150:0/26125074 >> 172.21.15.150:0/26125074 conn(0x7f3848073480 msgr2=0x7f3848073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:45.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.348+0000 7f384e800640 1 -- 172.21.15.150:0/26125074 shutdown_connections 2024-04-15T15:24:45.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.348+0000 7f384e800640 1 -- 172.21.15.150:0/26125074 wait complete. 2024-04-15T15:24:45.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.350+0000 7f384e800640 1 Processor -- start 2024-04-15T15:24:45.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.350+0000 7f384e800640 1 -- start start 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3830003f30 0x7f3848157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3848158510 0x7f3848158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384e800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384815af10 0x7f384815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384e800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38480cf410 con 0x7f3848158510 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384e800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38480cf290 con 0x7f384815af10 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384e800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f38480cf590 con 0x7f3830003f30 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3848158510 0x7f3848158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3848158510 0x7f3848158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35654/0 (socket says 172.21.15.150:35654) 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.351+0000 7f384ca00640 1 -- 172.21.15.150:0/937664416 learned_addr learned my addr 172.21.15.150:0/937664416 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:45.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.352+0000 7f384d400640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3830003f30 0x7f3848157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.352+0000 7f384de00640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384815af10 0x7f384815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.352+0000 7f384ca00640 1 -- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3830003f30 msgr2=0x7f3848157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.352+0000 7f384ca00640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3830003f30 0x7f3848157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.352+0000 7f384ca00640 1 -- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384815af10 msgr2=0x7f384815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.352+0000 7f384ca00640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384815af10 0x7f384815b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.352+0000 7f384ca00640 1 -- 172.21.15.150:0/937664416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38480ce720 con 0x7f3848158510 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.352+0000 7f384ca00640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3848158510 0x7f3848158990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3834000c00 tx=0x7f3834004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.353+0000 7f3845600640 1 -- 172.21.15.150:0/937664416 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834039050 con 0x7f3848158510 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.353+0000 7f3845600640 1 -- 172.21.15.150:0/937664416 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3834045070 con 0x7f3848158510 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.353+0000 7f3845600640 1 -- 172.21.15.150:0/937664416 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38340044e0 con 0x7f3848158510 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.353+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38480c8440 con 0x7f3848158510 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.353+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38481590f0 con 0x7f3848158510 2024-04-15T15:24:45.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.354+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38480755c0 con 0x7f3848158510 2024-04-15T15:24:45.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.355+0000 7f3845600640 1 -- 172.21.15.150:0/937664416 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3834003a50 con 0x7f3848158510 2024-04-15T15:24:45.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.356+0000 7f3845600640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f382806c920 0x7f382806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.356+0000 7f3845600640 1 -- 172.21.15.150:0/937664416 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f38340addf0 con 0x7f3848158510 2024-04-15T15:24:45.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.356+0000 7f384d400640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f382806c920 0x7f382806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.357+0000 7f384d400640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f382806c920 0x7f382806ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f38480d0d30 tx=0x7f383c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:45.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.364+0000 7f3845600640 1 -- 172.21.15.150:0/937664416 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38480ce720 con 0x7f3848158510 2024-04-15T15:24:45.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.564+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3848073ae0 con 0x7f3848158510 2024-04-15T15:24:45.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.569+0000 7f3845600640 1 -- 172.21.15.150:0/937664416 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f383407c160 con 0x7f3848158510 2024-04-15T15:24:45.570 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:45.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f382806c920 msgr2=0x7f382806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f382806c920 0x7f382806ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f38480d0d30 tx=0x7f383c009040 comp rx=0 tx=0).stop 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3848158510 msgr2=0x7f3848158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3848158510 0x7f3848158990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3834000c00 tx=0x7f3834004730 comp rx=0 tx=0).stop 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 shutdown_connections 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f384815af10 0x7f384815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3848158510 0x7f3848158990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f382806c920 0x7f382806ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 --2- 172.21.15.150:0/937664416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3830003f30 0x7f3848157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 >> 172.21.15.150:0/937664416 conn(0x7f3848073480 msgr2=0x7f384815ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 shutdown_connections 2024-04-15T15:24:45.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.572+0000 7f384e800640 1 -- 172.21.15.150:0/937664416 wait complete. 2024-04-15T15:24:45.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:45.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.783+0000 7f6479200640 1 Processor -- start 2024-04-15T15:24:45.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.783+0000 7f6479200640 1 -- start start 2024-04-15T15:24:45.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.784+0000 7f6479200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f64740d3260 0x7f64740b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.784+0000 7f6479200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6474162dd0 con 0x7f647415c680 2024-04-15T15:24:45.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.784+0000 7f6479200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64741630b0 con 0x7f6474162730 2024-04-15T15:24:45.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.784+0000 7f6479200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f6474162b90 con 0x7f64740d3260 2024-04-15T15:24:45.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.785+0000 7f6473e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f64740d3260 0x7f64740b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.785+0000 7f6473400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f647415c680 0x7f647415f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52960/0 (socket says 172.21.15.150:52960) 2024-04-15T15:24:45.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.785+0000 7f6473400640 1 -- 172.21.15.150:0/1175878565 learned_addr learned my addr 172.21.15.150:0/1175878565 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:45.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.785+0000 7f6473e00640 1 -- 172.21.15.150:0/1175878565 >> v1:172.21.15.187:6789/0 conn(0x7f6474162730 legacy=0x7f64740d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:45.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.785+0000 7f6473e00640 1 -- 172.21.15.150:0/1175878565 >> v1:172.21.15.150:6789/0 conn(0x7f647415c680 legacy=0x7f647415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.785+0000 7f6473e00640 1 -- 172.21.15.150:0/1175878565 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6474163390 con 0x7f64740d3260 2024-04-15T15:24:45.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.785+0000 7f6472000640 1 -- 172.21.15.150:0/1175878565 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100167759 0 0) 0x7f6474162dd0 con 0x7f647415c680 2024-04-15T15:24:45.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.786+0000 7f6473e00640 1 --2- 172.21.15.150:0/1175878565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f64740b6c70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f646c009d90 tx=0x7f646c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36961a2396c4740d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:45.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.786+0000 7f6472000640 1 -- 172.21.15.150:0/1175878565 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c039070 con 0x7f64740d3260 2024-04-15T15:24:45.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.786+0000 7f6472000640 1 -- 172.21.15.150:0/1175878565 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f646c02cc60 con 0x7f64740d3260 2024-04-15T15:24:45.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.786+0000 7f6472000640 1 -- 172.21.15.150:0/1175878565 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c034660 con 0x7f64740d3260 2024-04-15T15:24:45.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.787+0000 7f6479200640 1 -- 172.21.15.150:0/1175878565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 msgr2=0x7f64740b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.787+0000 7f6479200640 1 --2- 172.21.15.150:0/1175878565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f64740b6c70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f646c009d90 tx=0x7f646c02beb0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.787+0000 7f6479200640 1 -- 172.21.15.150:0/1175878565 shutdown_connections 2024-04-15T15:24:45.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.787+0000 7f6479200640 1 --2- 172.21.15.150:0/1175878565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f64740b6c70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.787+0000 7f6479200640 1 -- 172.21.15.150:0/1175878565 >> 172.21.15.150:0/1175878565 conn(0x7f6474073480 msgr2=0x7f6474073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:45.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.787+0000 7f6479200640 1 -- 172.21.15.150:0/1175878565 shutdown_connections 2024-04-15T15:24:45.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.787+0000 7f6479200640 1 -- 172.21.15.150:0/1175878565 wait complete. 2024-04-15T15:24:45.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.789+0000 7f6479200640 1 Processor -- start 2024-04-15T15:24:45.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.789+0000 7f6479200640 1 -- start start 2024-04-15T15:24:45.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6479200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f6474151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6479200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f647415c680 0x7f64740ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6479200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6474162730 0x7f64740cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6479200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f647415ff60 con 0x7f64740d3260 2024-04-15T15:24:45.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6479200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f647415ca60 con 0x7f647415c680 2024-04-15T15:24:45.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6479200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64740d1770 con 0x7f6474162730 2024-04-15T15:24:45.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6473400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f6474151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6473400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f6474151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35670/0 (socket says 172.21.15.150:35670) 2024-04-15T15:24:45.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6473400640 1 -- 172.21.15.150:0/3034405351 learned_addr learned my addr 172.21.15.150:0/3034405351 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6473400640 1 -- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6474162730 msgr2=0x7f64740cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6473400640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6474162730 0x7f64740cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6473400640 1 -- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f647415c680 msgr2=0x7f64740ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6473400640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f647415c680 0x7f64740ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.790+0000 7f6473400640 1 -- 172.21.15.150:0/3034405351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647415c1f0 con 0x7f64740d3260 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.791+0000 7f6473400640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f6474151be0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f646400c4d0 tx=0x7f646400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.791+0000 7f646b400640 1 -- 172.21.15.150:0/3034405351 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464018070 con 0x7f64740d3260 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.791+0000 7f646b400640 1 -- 172.21.15.150:0/3034405351 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64640074e0 con 0x7f64740d3260 2024-04-15T15:24:45.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.791+0000 7f646b400640 1 -- 172.21.15.150:0/3034405351 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464004a20 con 0x7f64740d3260 2024-04-15T15:24:45.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.791+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64740c9ac0 con 0x7f64740d3260 2024-04-15T15:24:45.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.791+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64740ca080 con 0x7f64740d3260 2024-04-15T15:24:45.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.792+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64740755c0 con 0x7f64740d3260 2024-04-15T15:24:45.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.793+0000 7f646b400640 1 -- 172.21.15.150:0/3034405351 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6464004bc0 con 0x7f64740d3260 2024-04-15T15:24:45.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.794+0000 7f646b400640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f645406c8f0 0x7f645406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:45.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.794+0000 7f646b400640 1 -- 172.21.15.150:0/3034405351 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f646408ae50 con 0x7f64740d3260 2024-04-15T15:24:45.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.795+0000 7f6472a00640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f645406c8f0 0x7f645406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:45.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.796+0000 7f6472a00640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f645406c8f0 0x7f645406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:45.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.799+0000 7f646b400640 1 -- 172.21.15.150:0/3034405351 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6464059260 con 0x7f64740d3260 2024-04-15T15:24:45.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.986+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f647406dbe0 con 0x7f64740d3260 2024-04-15T15:24:45.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.987+0000 7f646b400640 1 -- 172.21.15.150:0/3034405351 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f6464058b60 con 0x7f64740d3260 2024-04-15T15:24:45.989 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f645406c8f0 msgr2=0x7f645406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f645406c8f0 0x7f645406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460006bc0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 msgr2=0x7f6474151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f6474151be0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f646400c4d0 tx=0x7f646400c990 comp rx=0 tx=0).stop 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 shutdown_connections 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f645406c8f0 0x7f645406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6474162730 0x7f64740cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f647415c680 0x7f64740ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 --2- 172.21.15.150:0/3034405351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64740d3260 0x7f6474151be0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:45.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 >> 172.21.15.150:0/3034405351 conn(0x7f6474073480 msgr2=0x7f647415f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:45.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 shutdown_connections 2024-04-15T15:24:45.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:45.990+0000 7f6479200640 1 -- 172.21.15.150:0/3034405351 wait complete. 2024-04-15T15:24:46.011 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:46.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.208+0000 7fc21f200640 1 Processor -- start 2024-04-15T15:24:46.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.208+0000 7fc21f200640 1 -- start start 2024-04-15T15:24:46.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.209+0000 7fc21f200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc220158f10 0x7fc22006f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200b3dc0 0x7fc22006f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21f200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22015f230 con 0x7fc2200cd4e0 2024-04-15T15:24:46.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21f200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fc2200ce040 con 0x7fc220158f10 2024-04-15T15:24:46.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc22015f4d0 con 0x7fc2200b3dc0 2024-04-15T15:24:46.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21de00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc220158f10 0x7fc22006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21de00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc220158f10 0x7fc22006f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35676/0 (socket says 172.21.15.150:35676) 2024-04-15T15:24:46.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21de00640 1 -- 172.21.15.150:0/1269590797 learned_addr learned my addr 172.21.15.150:0/1269590797 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:46.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21d400640 1 --2- 172.21.15.150:0/1269590797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200b3dc0 0x7fc22006f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21de00640 1 -- 172.21.15.150:0/1269590797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200b3dc0 msgr2=0x7fc22006f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:46.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21de00640 1 --2- 172.21.15.150:0/1269590797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200b3dc0 0x7fc22006f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21de00640 1 -- 172.21.15.150:0/1269590797 >> v1:172.21.15.187:6789/0 conn(0x7fc2200cd4e0 legacy=0x7fc2200cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:46.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.210+0000 7fc21de00640 1 -- 172.21.15.150:0/1269590797 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22015f7b0 con 0x7fc220158f10 2024-04-15T15:24:46.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.212+0000 7fc21de00640 1 --2- 172.21.15.150:0/1269590797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc22006f030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc21000bb10 tx=0x7fc21002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aafee599ddc7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:46.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.212+0000 7fc21ca00640 1 -- 172.21.15.150:0/1269590797 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21003b070 con 0x7fc220158f10 2024-04-15T15:24:46.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.212+0000 7fc21ca00640 1 -- 172.21.15.150:0/1269590797 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21002ede0 con 0x7fc220158f10 2024-04-15T15:24:46.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.212+0000 7fc21ca00640 1 -- 172.21.15.150:0/1269590797 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210036660 con 0x7fc220158f10 2024-04-15T15:24:46.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.213+0000 7fc21f200640 1 -- 172.21.15.150:0/1269590797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 msgr2=0x7fc22006f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.213+0000 7fc21f200640 1 --2- 172.21.15.150:0/1269590797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc22006f030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc21000bb10 tx=0x7fc21002e030 comp rx=0 tx=0).stop 2024-04-15T15:24:46.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.213+0000 7fc21f200640 1 -- 172.21.15.150:0/1269590797 shutdown_connections 2024-04-15T15:24:46.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.213+0000 7fc21f200640 1 --2- 172.21.15.150:0/1269590797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200b3dc0 0x7fc22006f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.213+0000 7fc21f200640 1 --2- 172.21.15.150:0/1269590797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc22006f030 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.213+0000 7fc21f200640 1 -- 172.21.15.150:0/1269590797 >> 172.21.15.150:0/1269590797 conn(0x7fc22006b930 msgr2=0x7fc22006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:46.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.213+0000 7fc21f200640 1 -- 172.21.15.150:0/1269590797 shutdown_connections 2024-04-15T15:24:46.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.213+0000 7fc21f200640 1 -- 172.21.15.150:0/1269590797 wait complete. 2024-04-15T15:24:46.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.214+0000 7fc21f200640 1 Processor -- start 2024-04-15T15:24:46.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.215+0000 7fc21f200640 1 -- start start 2024-04-15T15:24:46.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.215+0000 7fc21f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2200b3dc0 0x7fc2201596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.215+0000 7fc21f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200cd4e0 0x7fc220149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc2200b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201592f0 con 0x7fc220158f10 2024-04-15T15:24:46.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc22015fd70 con 0x7fc2200b3dc0 2024-04-15T15:24:46.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc220159470 con 0x7fc2200cd4e0 2024-04-15T15:24:46.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc2200b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc2200b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35686/0 (socket says 172.21.15.150:35686) 2024-04-15T15:24:46.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21e800640 1 -- 172.21.15.150:0/3531664673 learned_addr learned my addr 172.21.15.150:0/3531664673 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21de00640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2200b3dc0 0x7fc2201596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21e800640 1 -- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200cd4e0 msgr2=0x7fc220149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21d400640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200cd4e0 0x7fc220149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21e800640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200cd4e0 0x7fc220149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21e800640 1 -- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2200b3dc0 msgr2=0x7fc2201596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21e800640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2200b3dc0 0x7fc2201596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21e800640 1 -- 172.21.15.150:0/3531664673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2201460e0 con 0x7fc220158f10 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21d400640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200cd4e0 0x7fc220149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.216+0000 7fc21de00640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2200b3dc0 0x7fc2201596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.217+0000 7fc21e800640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc2200b4210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c780 tx=0x7fc21800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:46.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.217+0000 7fc216000640 1 -- 172.21.15.150:0/3531664673 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218004b60 con 0x7fc220158f10 2024-04-15T15:24:46.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.217+0000 7fc216000640 1 -- 172.21.15.150:0/3531664673 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc218004d00 con 0x7fc220158f10 2024-04-15T15:24:46.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.217+0000 7fc216000640 1 -- 172.21.15.150:0/3531664673 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21800aa90 con 0x7fc220158f10 2024-04-15T15:24:46.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.217+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2201463c0 con 0x7fc220158f10 2024-04-15T15:24:46.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.217+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc220146980 con 0x7fc220158f10 2024-04-15T15:24:46.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.218+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc220146d80 con 0x7fc220158f10 2024-04-15T15:24:46.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.219+0000 7fc216000640 1 -- 172.21.15.150:0/3531664673 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc21800ac30 con 0x7fc220158f10 2024-04-15T15:24:46.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.220+0000 7fc216000640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc20006c8f0 0x7fc20006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.221+0000 7fc216000640 1 -- 172.21.15.150:0/3531664673 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fc21808bfa0 con 0x7fc220158f10 2024-04-15T15:24:46.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.221+0000 7fc21de00640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc20006c8f0 0x7fc20006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.222+0000 7fc21de00640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc20006c8f0 0x7fc20006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc210038000 tx=0x7fc210008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:46.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.224+0000 7fc216000640 1 -- 172.21.15.150:0/3531664673 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc218020070 con 0x7fc220158f10 2024-04-15T15:24:46.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.416+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2200755c0 con 0x7fc220158f10 2024-04-15T15:24:46.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.417+0000 7fc216000640 1 -- 172.21.15.150:0/3531664673 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fc21805a280 con 0x7fc220158f10 2024-04-15T15:24:46.418 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:46.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc20006c8f0 msgr2=0x7fc20006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc20006c8f0 0x7fc20006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc210038000 tx=0x7fc210008040 comp rx=0 tx=0).stop 2024-04-15T15:24:46.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 msgr2=0x7fc2200b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc2200b4210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c780 tx=0x7fc21800cc40 comp rx=0 tx=0).stop 2024-04-15T15:24:46.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 shutdown_connections 2024-04-15T15:24:46.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220158f10 0x7fc2200b4210 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc20006c8f0 0x7fc20006edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2200cd4e0 0x7fc220149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 --2- 172.21.15.150:0/3531664673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2200b3dc0 0x7fc2201596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.419+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 >> 172.21.15.150:0/3531664673 conn(0x7fc22006b930 msgr2=0x7fc22014df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:46.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.420+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 shutdown_connections 2024-04-15T15:24:46.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.420+0000 7fc21f200640 1 -- 172.21.15.150:0/3531664673 wait complete. 2024-04-15T15:24:46.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:24:46.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.640+0000 7f1b2be00640 1 Processor -- start 2024-04-15T15:24:46.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.640+0000 7f1b2be00640 1 -- start start 2024-04-15T15:24:46.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1b2c15eab0 0x7f1b2c162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 0x7f1b2c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2be00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b2c0b78f0 con 0x7f1b2c0d1a90 2024-04-15T15:24:46.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c0d2e50 con 0x7f1b2c15eab0 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c163010 con 0x7f1b2c162b70 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2b400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1b2c0d1a90 0x7f1b2c0b6240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:52976/0 (socket says 172.21.15.150:52976) 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2b400640 1 -- 172.21.15.150:0/3490559062 learned_addr learned my addr 172.21.15.150:0/3490559062 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2a000640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 0x7f1b2c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b2aa00640 1 --2- 172.21.15.150:0/3490559062 >> v2:172.21.15.187:3300/0 conn(0x7f1b2c15eab0 0x7f1b2c162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909567568 0 0) 0x7f1b24001490 con 0x7f1b2c0d1a90 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.642+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b10003520 con 0x7f1b2c0d1a90 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4098772515 0 0) 0x7f1b10003520 con 0x7f1b2c0d1a90 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 msgr2=0x7f1b2c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 0x7f1b2c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 >> v2:172.21.15.187:3300/0 conn(0x7f1b2c15eab0 msgr2=0x7f1b2c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 --2- 172.21.15.150:0/3490559062 >> v2:172.21.15.187:3300/0 conn(0x7f1b2c15eab0 0x7f1b2c162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c1632f0 con 0x7f1b2c0d1a90 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f1b24001bc0 con 0x7f1b2c0d1a90 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 >> v1:172.21.15.150:6789/0 conn(0x7f1b2c0d1a90 legacy=0x7f1b2c0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b100039b0 0x7f1b10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b10004800 0x7f1b10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 0x7f1b100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c0d2e50 con 0x7f1b10004800 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c163010 con 0x7f1b100039b0 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1000b9c0 con 0x7f1b2c162b70 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.643+0000 7f1b2a000640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b10004800 0x7f1b10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2aa00640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b100039b0 0x7f1b10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2b400640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 0x7f1b100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2a000640 1 -- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 msgr2=0x7f1b100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2a000640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 0x7f1b100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2a000640 1 -- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b100039b0 msgr2=0x7f1b10003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2a000640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b100039b0 0x7f1b10003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2a000640 1 -- 172.21.15.150:0/3490559062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b10003520 con 0x7f1b10004800 2024-04-15T15:24:46.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2aa00640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b100039b0 0x7f1b10003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.644+0000 7f1b2b400640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c162b70 0x7f1b100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.645+0000 7f1b2a000640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b10004800 0x7f1b10004c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009ac0 tx=0x7f1b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c39d74bc20c3e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.645+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18039070 con 0x7f1b10004800 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.645+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1802c6b0 con 0x7f1b10004800 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.645+0000 7f1b29600640 1 -- 172.21.15.150:0/3490559062 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18009690 con 0x7f1b10004800 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.646+0000 7f1b2be00640 1 -- 172.21.15.150:0/3490559062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b10004800 msgr2=0x7f1b10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.646+0000 7f1b2be00640 1 --2- 172.21.15.150:0/3490559062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b10004800 0x7f1b10004c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009ac0 tx=0x7f1b10005830 comp rx=0 tx=0).stop 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.646+0000 7f1b2b400640 1 -- 172.21.15.150:0/3490559062 reap_dead start 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.646+0000 7f1b2be00640 1 -- 172.21.15.150:0/3490559062 shutdown_connections 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.646+0000 7f1b2be00640 1 -- 172.21.15.150:0/3490559062 >> 172.21.15.150:0/3490559062 conn(0x7f1b2c073240 msgr2=0x7f1b2c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.646+0000 7f1b2be00640 1 -- 172.21.15.150:0/3490559062 shutdown_connections 2024-04-15T15:24:46.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.646+0000 7f1b2be00640 1 -- 172.21.15.150:0/3490559062 wait complete. 2024-04-15T15:24:46.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.648+0000 7f1b2be00640 1 Processor -- start 2024-04-15T15:24:46.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.649+0000 7f1b2be00640 1 -- start start 2024-04-15T15:24:46.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b10004800 0x7f1b2c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2c0cdb90 0x7f1b2c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c0ce010 0x7f1b2c1488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c0ba590 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c15ee90 con 0x7f1b10004800 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c0ba710 con 0x7f1b2c0ce010 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2c0cdb90 0x7f1b2c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2c0cdb90 0x7f1b2c15a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35700/0 (socket says 172.21.15.150:35700) 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.650+0000 7f1b2a000640 1 -- 172.21.15.150:0/3771801712 learned_addr learned my addr 172.21.15.150:0/3771801712 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2b400640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c0ce010 0x7f1b2c1488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2aa00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b10004800 0x7f1b2c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2a000640 1 -- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c0ce010 msgr2=0x7f1b2c1488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2a000640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c0ce010 0x7f1b2c1488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2a000640 1 -- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b10004800 msgr2=0x7f1b2c0c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2a000640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b10004800 0x7f1b2c0c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2a000640 1 -- 172.21.15.150:0/3771801712 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c162160 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2b400640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c0ce010 0x7f1b2c1488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2a000640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2c0cdb90 0x7f1b2c15a920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1b18037040 tx=0x7f1b18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b22a00640 1 -- 172.21.15.150:0/3771801712 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18039050 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b22a00640 1 -- 172.21.15.150:0/3771801712 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b18045070 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c15dde0 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.651+0000 7f1b22a00640 1 -- 172.21.15.150:0/3771801712 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18003a50 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.652+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c15db00 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.653+0000 7f1b22a00640 1 -- 172.21.15.150:0/3771801712 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b18003bf0 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.653+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c0755c0 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.654+0000 7f1b22a00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1b0c06c8f0 0x7f1b0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:46.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.654+0000 7f1b22a00640 1 -- 172.21.15.150:0/3771801712 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f1b180ad720 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.655+0000 7f1b2aa00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1b0c06c8f0 0x7f1b0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:46.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.656+0000 7f1b2aa00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1b0c06c8f0 0x7f1b0c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009a40 tx=0x7f1b1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:46.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.662+0000 7f1b22a00640 1 -- 172.21.15.150:0/3771801712 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b1804e020 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.664 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:24:46.670 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:24:46.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.869+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1b2c0d2460 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.869+0000 7f1b22a00640 1 -- 172.21.15.150:0/3771801712 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f1b1807bb60 con 0x7f1b2c0cdb90 2024-04-15T15:24:46.871 INFO:teuthology.orchestra.run.smithi150.stderr:listed 0 entries 2024-04-15T15:24:46.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1b0c06c8f0 msgr2=0x7f1b0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1b0c06c8f0 0x7f1b0c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009a40 tx=0x7f1b1c008040 comp rx=0 tx=0).stop 2024-04-15T15:24:46.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2c0cdb90 msgr2=0x7f1b2c15a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2c0cdb90 0x7f1b2c15a920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1b18037040 tx=0x7f1b18004730 comp rx=0 tx=0).stop 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 shutdown_connections 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1b0c06c8f0 0x7f1b0c06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2c0ce010 0x7f1b2c1488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2c0cdb90 0x7f1b2c15a920 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 --2- 172.21.15.150:0/3771801712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b10004800 0x7f1b2c0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 >> 172.21.15.150:0/3771801712 conn(0x7f1b2c073240 msgr2=0x7f1b2c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 shutdown_connections 2024-04-15T15:24:46.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:46.872+0000 7f1b2be00640 1 -- 172.21.15.150:0/3771801712 wait complete. 2024-04-15T15:24:46.891 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-15T15:24:46.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:46.891 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-15T15:24:46.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:24:46.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:46.981 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:24:47.025 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:24:47.026 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:47.081 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:24:47.081 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:47.132 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:24:47.132 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:24:47.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:47.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.390+0000 7f0daae00640 1 Processor -- start 2024-04-15T15:24:47.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.390+0000 7f0daae00640 1 -- start start 2024-04-15T15:24:47.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.392+0000 7f0daae00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0da4148130 0x7f0da415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.392+0000 7f0daae00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0da4072660 0x7f0da40af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.392+0000 7f0daae00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da4162940 con 0x7f0da4162380 2024-04-15T15:24:47.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.392+0000 7f0daae00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0da4148510 con 0x7f0da4148130 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.392+0000 7f0daae00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0da415fd70 con 0x7f0da4072660 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.392+0000 7f0da9a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0da4148130 0x7f0da415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.392+0000 7f0da9000640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0da4162380 0x7f0da4071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56640/0 (socket says 172.21.15.150:56640) 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.392+0000 7f0da9000640 1 -- 172.21.15.150:0/3333347605 learned_addr learned my addr 172.21.15.150:0/3333347605 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.393+0000 7f0daa400640 1 --2- 172.21.15.150:0/3333347605 >> v2:172.21.15.187:3300/0 conn(0x7f0da4072660 0x7f0da40af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.393+0000 7f0da9a00640 1 -- 172.21.15.150:0/3333347605 >> v2:172.21.15.187:3300/0 conn(0x7f0da4072660 msgr2=0x7f0da40af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.393+0000 7f0da9a00640 1 --2- 172.21.15.150:0/3333347605 >> v2:172.21.15.187:3300/0 conn(0x7f0da4072660 0x7f0da40af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.393+0000 7f0da9a00640 1 -- 172.21.15.150:0/3333347605 >> v1:172.21.15.150:6789/0 conn(0x7f0da4162380 legacy=0x7f0da4071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.393+0000 7f0da9a00640 1 -- 172.21.15.150:0/3333347605 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da4162c20 con 0x7f0da4148130 2024-04-15T15:24:47.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.394+0000 7f0da9a00640 1 --2- 172.21.15.150:0/3333347605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da415f830 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0da406ff10 tx=0x7f0d9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a675c146617c46 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:47.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.394+0000 7f0da3e00640 1 -- 172.21.15.150:0/3333347605 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d980315a0 con 0x7f0da4148130 2024-04-15T15:24:47.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.394+0000 7f0da3e00640 1 -- 172.21.15.150:0/3333347605 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d98037070 con 0x7f0da4148130 2024-04-15T15:24:47.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.394+0000 7f0da3e00640 1 -- 172.21.15.150:0/3333347605 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98031960 con 0x7f0da4148130 2024-04-15T15:24:47.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.395+0000 7f0daae00640 1 -- 172.21.15.150:0/3333347605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 msgr2=0x7f0da415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.395+0000 7f0daae00640 1 --2- 172.21.15.150:0/3333347605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da415f830 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0da406ff10 tx=0x7f0d9802b840 comp rx=0 tx=0).stop 2024-04-15T15:24:47.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.395+0000 7f0daae00640 1 -- 172.21.15.150:0/3333347605 shutdown_connections 2024-04-15T15:24:47.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.395+0000 7f0daae00640 1 --2- 172.21.15.150:0/3333347605 >> v2:172.21.15.187:3300/0 conn(0x7f0da4072660 0x7f0da40af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.395+0000 7f0daae00640 1 --2- 172.21.15.150:0/3333347605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da415f830 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.395+0000 7f0daae00640 1 -- 172.21.15.150:0/3333347605 >> 172.21.15.150:0/3333347605 conn(0x7f0da406b930 msgr2=0x7f0da406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:47.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.395+0000 7f0daae00640 1 -- 172.21.15.150:0/3333347605 shutdown_connections 2024-04-15T15:24:47.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.395+0000 7f0daae00640 1 -- 172.21.15.150:0/3333347605 wait complete. 2024-04-15T15:24:47.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.397+0000 7f0daae00640 1 Processor -- start 2024-04-15T15:24:47.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.397+0000 7f0daae00640 1 -- start start 2024-04-15T15:24:47.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0daae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da4072660 0x7f0da4161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0daae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da40ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0daae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da4162380 0x7f0da41f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0daae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40aeb90 con 0x7f0da4148130 2024-04-15T15:24:47.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0daae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0da41631e0 con 0x7f0da4162380 2024-04-15T15:24:47.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0daae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0da40aed10 con 0x7f0da4072660 2024-04-15T15:24:47.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0da9000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da40ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0da9000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da40ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33492/0 (socket says 172.21.15.150:33492) 2024-04-15T15:24:47.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0da9000640 1 -- 172.21.15.150:0/1042802688 learned_addr learned my addr 172.21.15.150:0/1042802688 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0daa400640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da4162380 0x7f0da41f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0da9000640 1 -- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da4072660 msgr2=0x7f0da4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0da9000640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da4072660 0x7f0da4161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0da9000640 1 -- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da4162380 msgr2=0x7f0da41f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0da9000640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da4162380 0x7f0da41f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.398+0000 7f0da9000640 1 -- 172.21.15.150:0/1042802688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da40af500 con 0x7f0da4148130 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.399+0000 7f0da9000640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da40ce300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c3c0 tx=0x7f0d9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.399+0000 7f0da1600640 1 -- 172.21.15.150:0/1042802688 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90017070 con 0x7f0da4148130 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.399+0000 7f0da1600640 1 -- 172.21.15.150:0/1042802688 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d90005ce0 con 0x7f0da4148130 2024-04-15T15:24:47.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.399+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da40af7e0 con 0x7f0da4148130 2024-04-15T15:24:47.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.399+0000 7f0da1600640 1 -- 172.21.15.150:0/1042802688 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d900072f0 con 0x7f0da4148130 2024-04-15T15:24:47.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.399+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da40afda0 con 0x7f0da4148130 2024-04-15T15:24:47.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.400+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da40755c0 con 0x7f0da4148130 2024-04-15T15:24:47.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.401+0000 7f0da1600640 1 -- 172.21.15.150:0/1042802688 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d9001a070 con 0x7f0da4148130 2024-04-15T15:24:47.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.402+0000 7f0da1600640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8406c8f0 0x7f0d8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.403+0000 7f0da1600640 1 -- 172.21.15.150:0/1042802688 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f0d9008aff0 con 0x7f0da4148130 2024-04-15T15:24:47.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.403+0000 7f0da9a00640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8406c8f0 0x7f0d8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.404+0000 7f0da9a00640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8406c8f0 0x7f0d8406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d98036000 tx=0x7f0d98030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:47.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.407+0000 7f0da1600640 1 -- 172.21.15.150:0/1042802688 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d900592d0 con 0x7f0da4148130 2024-04-15T15:24:47.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.606+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0da406f6d0 con 0x7f0da4148130 2024-04-15T15:24:47.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.608+0000 7f0da1600640 1 -- 172.21.15.150:0/1042802688 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16997 (secure 0 0 0) 0x7f0d90058bd0 con 0x7f0da4148130 2024-04-15T15:24:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:47.610 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:36.500392+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:47.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.610+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8406c8f0 msgr2=0x7f0d8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.610+0000 7f0daae00640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8406c8f0 0x7f0d8406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d98036000 tx=0x7f0d98030000 comp rx=0 tx=0).stop 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.610+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 msgr2=0x7f0da40ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.610+0000 7f0daae00640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da40ce300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c3c0 tx=0x7f0d9000c880 comp rx=0 tx=0).stop 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.611+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 shutdown_connections 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.611+0000 7f0daae00640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da4162380 0x7f0da41f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.611+0000 7f0daae00640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da4148130 0x7f0da40ce300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.611+0000 7f0daae00640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8406c8f0 0x7f0d8406edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.611+0000 7f0daae00640 1 --2- 172.21.15.150:0/1042802688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da4072660 0x7f0da4161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.611+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 >> 172.21.15.150:0/1042802688 conn(0x7f0da406b930 msgr2=0x7f0da40b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.611+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 shutdown_connections 2024-04-15T15:24:47.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.611+0000 7f0daae00640 1 -- 172.21.15.150:0/1042802688 wait complete. 2024-04-15T15:24:47.632 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:47.632 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:47.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.840+0000 7fb97dc00640 1 Processor -- start 2024-04-15T15:24:47.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.840+0000 7fb97dc00640 1 -- start start 2024-04-15T15:24:47.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.842+0000 7fb97dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9780d18d0 0x7fb9780d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.842+0000 7fb97dc00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb97815f130 con 0x7fb97815ed50 2024-04-15T15:24:47.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.842+0000 7fb97dc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9780d3210 con 0x7fb9780cf1a0 2024-04-15T15:24:47.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.842+0000 7fb97dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb9780b7e40 con 0x7fb9780d18d0 2024-04-15T15:24:47.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.842+0000 7fb976a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fb97815ed50 0x7fb9780d2b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56654/0 (socket says 172.21.15.150:56654) 2024-04-15T15:24:47.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.842+0000 7fb976a00640 1 -- 172.21.15.150:0/833545088 learned_addr learned my addr 172.21.15.150:0/833545088 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb96e000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9780d18d0 0x7fb9780d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141048577 0 0) 0x7fb97815f130 con 0x7fb97815ed50 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb95c003520 con 0x7fb97815ed50 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 932249507 0 0) 0x7fb95c003520 con 0x7fb97815ed50 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9780d18d0 msgr2=0x7fb9780d1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9780d18d0 0x7fb9780d1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 >> v1:172.21.15.187:6789/0 conn(0x7fb9780cf1a0 legacy=0x7fb9780cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9780d34e0 con 0x7fb97815ed50 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fb968001670 con 0x7fb97815ed50 2024-04-15T15:24:47.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 >> v1:172.21.15.150:6789/0 conn(0x7fb97815ed50 legacy=0x7fb9780d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9780d18d0 0x7fb95c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb95c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb95c004d40 0x7fb95c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb9780b7e40 con 0x7fb95c004010 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c00bc00 con 0x7fb95c004d40 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.843+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb95c00be00 con 0x7fb9780d18d0 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb96e000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb95c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb977400640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb95c004d40 0x7fb95c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb976a00640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9780d18d0 0x7fb95c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb96e000640 1 -- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9780d18d0 msgr2=0x7fb95c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb96e000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9780d18d0 0x7fb95c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb96e000640 1 -- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb95c004d40 msgr2=0x7fb95c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb96e000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb95c004d40 0x7fb95c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb96e000640 1 -- 172.21.15.150:0/833545088 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c003520 con 0x7fb95c004010 2024-04-15T15:24:47.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.844+0000 7fb977400640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb95c004d40 0x7fb95c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.845+0000 7fb96e000640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb95c004800 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb96000ba30 tx=0x7fb95c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca28c7594e07f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.845+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960036020 con 0x7fb95c004010 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.845+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb960032640 con 0x7fb95c004010 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.845+0000 7fb976000640 1 -- 172.21.15.150:0/833545088 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96000b600 con 0x7fb95c004010 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.846+0000 7fb97dc00640 1 -- 172.21.15.150:0/833545088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 msgr2=0x7fb95c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.846+0000 7fb97dc00640 1 --2- 172.21.15.150:0/833545088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb95c004800 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb96000ba30 tx=0x7fb95c005830 comp rx=0 tx=0).stop 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.846+0000 7fb977400640 1 -- 172.21.15.150:0/833545088 reap_dead start 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.846+0000 7fb97dc00640 1 -- 172.21.15.150:0/833545088 shutdown_connections 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.846+0000 7fb97dc00640 1 -- 172.21.15.150:0/833545088 >> 172.21.15.150:0/833545088 conn(0x7fb97806d400 msgr2=0x7fb97806d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.846+0000 7fb97dc00640 1 -- 172.21.15.150:0/833545088 shutdown_connections 2024-04-15T15:24:47.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.846+0000 7fb97dc00640 1 -- 172.21.15.150:0/833545088 wait complete. 2024-04-15T15:24:47.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.848+0000 7fb97dc00640 1 Processor -- start 2024-04-15T15:24:47.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.848+0000 7fb97dc00640 1 -- start start 2024-04-15T15:24:47.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.849+0000 7fb97dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb978161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.849+0000 7fb97dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb978161840 0x7fb9780ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.849+0000 7fb97dc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9780aba10 0x7fb97814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.849+0000 7fb97dc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb9780b79d0 con 0x7fb95c004010 2024-04-15T15:24:47.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.849+0000 7fb97dc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb9780d3aa0 con 0x7fb9780aba10 2024-04-15T15:24:47.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.849+0000 7fb97dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb9780b7b50 con 0x7fb978161840 2024-04-15T15:24:47.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb978161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb977400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9780aba10 0x7fb97814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb96e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb978161840 0x7fb9780ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb978161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33510/0 (socket says 172.21.15.150:33510) 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 -- 172.21.15.150:0/1031140846 learned_addr learned my addr 172.21.15.150:0/1031140846 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 -- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb978161840 msgr2=0x7fb9780ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb978161840 0x7fb9780ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 -- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9780aba10 msgr2=0x7fb97814ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9780aba10 0x7fb97814ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 -- 172.21.15.150:0/1031140846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9780aca20 con 0x7fb95c004010 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.850+0000 7fb976a00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb978161300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb96800d7e0 tx=0x7fb96800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.851+0000 7fb96f400640 1 -- 172.21.15.150:0/1031140846 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968019070 con 0x7fb95c004010 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.851+0000 7fb96f400640 1 -- 172.21.15.150:0/1031140846 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9680074e0 con 0x7fb95c004010 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.851+0000 7fb96f400640 1 -- 172.21.15.150:0/1031140846 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968004a00 con 0x7fb95c004010 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.851+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9780ac200 con 0x7fb95c004010 2024-04-15T15:24:47.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.851+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97814e470 con 0x7fb95c004010 2024-04-15T15:24:47.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.852+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb978068c80 con 0x7fb95c004010 2024-04-15T15:24:47.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.853+0000 7fb96f400640 1 -- 172.21.15.150:0/1031140846 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb968004ba0 con 0x7fb95c004010 2024-04-15T15:24:47.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.854+0000 7fb96f400640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb95406c920 0x7fb95406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:47.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.854+0000 7fb96f400640 1 -- 172.21.15.150:0/1031140846 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fb96808b560 con 0x7fb95c004010 2024-04-15T15:24:47.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.854+0000 7fb96e000640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb95406c920 0x7fb95406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:47.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.856+0000 7fb96e000640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb95406c920 0x7fb95406ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb96000ba00 tx=0x7fb96003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:47.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:47.858+0000 7fb96f400640 1 -- 172.21.15.150:0/1031140846 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9780aca20 con 0x7fb95c004010 2024-04-15T15:24:48.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.057+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb978161c20 con 0x7fb95c004010 2024-04-15T15:24:48.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.058+0000 7fb96f400640 1 -- 172.21.15.150:0/1031140846 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fb9680591f0 con 0x7fb95c004010 2024-04-15T15:24:48.060 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb95406c920 msgr2=0x7fb95406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb95406c920 0x7fb95406ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb96000ba00 tx=0x7fb96003d040 comp rx=0 tx=0).stop 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 msgr2=0x7fb978161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb978161300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb96800d7e0 tx=0x7fb96800dca0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 shutdown_connections 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9780aba10 0x7fb97814ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb95406c920 0x7fb95406ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb978161840 0x7fb9780ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 --2- 172.21.15.150:0/1031140846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb95c004010 0x7fb978161300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 >> 172.21.15.150:0/1031140846 conn(0x7fb97806d400 msgr2=0x7fb9780cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 shutdown_connections 2024-04-15T15:24:48.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.061+0000 7fb97dc00640 1 -- 172.21.15.150:0/1031140846 wait complete. 2024-04-15T15:24:48.080 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:48.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.293+0000 7f442fe00640 1 Processor -- start 2024-04-15T15:24:48.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.293+0000 7f442fe00640 1 -- start start 2024-04-15T15:24:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.295+0000 7f442fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4428163520 0x7f442815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.295+0000 7f442fe00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f442815ffe0 con 0x7f4428072860 2024-04-15T15:24:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.295+0000 7f442fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4428162a40 con 0x7f4428145070 2024-04-15T15:24:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.295+0000 7f442fe00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f44280afde0 con 0x7f4428163520 2024-04-15T15:24:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.295+0000 7f442ca00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f4428072860 0x7f442815f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56658/0 (socket says 172.21.15.150:56658) 2024-04-15T15:24:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.295+0000 7f442de00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4428163520 0x7f442815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.295+0000 7f442ca00640 1 -- 172.21.15.150:0/3873430303 learned_addr learned my addr 172.21.15.150:0/3873430303 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.296+0000 7f442de00640 1 -- 172.21.15.150:0/3873430303 >> v1:172.21.15.187:6789/0 conn(0x7f4428145070 legacy=0x7f4428072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.296+0000 7f442de00640 1 -- 172.21.15.150:0/3873430303 >> v1:172.21.15.150:6789/0 conn(0x7f4428072860 legacy=0x7f442815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.296+0000 7f442de00640 1 -- 172.21.15.150:0/3873430303 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4428162d20 con 0x7f4428163520 2024-04-15T15:24:48.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.297+0000 7f442de00640 1 --2- 172.21.15.150:0/3873430303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428163520 0x7f442815f8b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f44200097d0 tx=0x7f442002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492a2e323bd64219 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.297+0000 7f4427e00640 1 -- 172.21.15.150:0/3873430303 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420031780 con 0x7f4428163520 2024-04-15T15:24:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.297+0000 7f4427e00640 1 -- 172.21.15.150:0/3873430303 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420037070 con 0x7f4428163520 2024-04-15T15:24:48.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.297+0000 7f4427e00640 1 -- 172.21.15.150:0/3873430303 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420031b40 con 0x7f4428163520 2024-04-15T15:24:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.298+0000 7f442fe00640 1 -- 172.21.15.150:0/3873430303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428163520 msgr2=0x7f442815f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.298+0000 7f442fe00640 1 --2- 172.21.15.150:0/3873430303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428163520 0x7f442815f8b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f44200097d0 tx=0x7f442002b820 comp rx=0 tx=0).stop 2024-04-15T15:24:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.298+0000 7f442fe00640 1 -- 172.21.15.150:0/3873430303 shutdown_connections 2024-04-15T15:24:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.298+0000 7f442fe00640 1 --2- 172.21.15.150:0/3873430303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428163520 0x7f442815f8b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.298+0000 7f442fe00640 1 -- 172.21.15.150:0/3873430303 >> 172.21.15.150:0/3873430303 conn(0x7f44280b2460 msgr2=0x7f44280b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.298+0000 7f442fe00640 1 -- 172.21.15.150:0/3873430303 shutdown_connections 2024-04-15T15:24:48.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.298+0000 7f442fe00640 1 -- 172.21.15.150:0/3873430303 wait complete. 2024-04-15T15:24:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.300+0000 7f442fe00640 1 Processor -- start 2024-04-15T15:24:48.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.300+0000 7f442fe00640 1 -- start start 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4428072860 0x7f442815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428145070 0x7f4428158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4428163520 0x7f4428158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4428145450 con 0x7f4428145070 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44281627f0 con 0x7f4428072860 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4428169b80 con 0x7f4428163520 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428145070 0x7f4428158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428145070 0x7f4428158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33530/0 (socket says 172.21.15.150:33530) 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.301+0000 7f442ca00640 1 -- 172.21.15.150:0/3263843733 learned_addr learned my addr 172.21.15.150:0/3263843733 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:48.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442d400640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4428072860 0x7f442815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442de00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4428163520 0x7f4428158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442ca00640 1 -- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4428163520 msgr2=0x7f4428158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442ca00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4428163520 0x7f4428158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442ca00640 1 -- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4428072860 msgr2=0x7f442815ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442ca00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4428072860 0x7f442815ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442ca00640 1 -- 172.21.15.150:0/3263843733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442815e760 con 0x7f4428145070 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442d400640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4428072860 0x7f442815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442de00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4428163520 0x7f4428158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f442ca00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428145070 0x7f4428158990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f441400c3c0 tx=0x7f441400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f4425600640 1 -- 172.21.15.150:0/3263843733 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414017070 con 0x7f4428145070 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f4425600640 1 -- 172.21.15.150:0/3263843733 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4414005ce0 con 0x7f4428145070 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.302+0000 7f4425600640 1 -- 172.21.15.150:0/3263843733 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44140072f0 con 0x7f4428145070 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.303+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442815ea40 con 0x7f4428145070 2024-04-15T15:24:48.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.303+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4428161fd0 con 0x7f4428145070 2024-04-15T15:24:48.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.304+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4428068c80 con 0x7f4428145070 2024-04-15T15:24:48.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.305+0000 7f4425600640 1 -- 172.21.15.150:0/3263843733 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f441401a070 con 0x7f4428145070 2024-04-15T15:24:48.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.306+0000 7f4425600640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f440806c920 0x7f440806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.306+0000 7f4425600640 1 -- 172.21.15.150:0/3263843733 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f441408af40 con 0x7f4428145070 2024-04-15T15:24:48.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.306+0000 7f442d400640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f440806c920 0x7f440806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.307+0000 7f442d400640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f440806c920 0x7f440806ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f441c0045b0 tx=0x7f441c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:48.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.310+0000 7f4425600640 1 -- 172.21.15.150:0/3263843733 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44140592b0 con 0x7f4428145070 2024-04-15T15:24:48.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.511+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f442806f8d0 con 0x7f4428145070 2024-04-15T15:24:48.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.512+0000 7f4425600640 1 -- 172.21.15.150:0/3263843733 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f4414058bb0 con 0x7f4428145070 2024-04-15T15:24:48.514 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:48.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f440806c920 msgr2=0x7f440806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f440806c920 0x7f440806ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f441c0045b0 tx=0x7f441c009040 comp rx=0 tx=0).stop 2024-04-15T15:24:48.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428145070 msgr2=0x7f4428158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428145070 0x7f4428158990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f441400c3c0 tx=0x7f441400c880 comp rx=0 tx=0).stop 2024-04-15T15:24:48.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 shutdown_connections 2024-04-15T15:24:48.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f440806c920 0x7f440806ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4428163520 0x7f4428158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.517 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4428145070 0x7f4428158990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 --2- 172.21.15.150:0/3263843733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4428072860 0x7f442815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 >> 172.21.15.150:0/3263843733 conn(0x7f44280b2460 msgr2=0x7f44280b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:48.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 shutdown_connections 2024-04-15T15:24:48.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.515+0000 7f442fe00640 1 -- 172.21.15.150:0/3263843733 wait complete. 2024-04-15T15:24:48.537 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:48.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.740+0000 7f72bca00640 1 Processor -- start 2024-04-15T15:24:48.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.740+0000 7f72bca00640 1 -- start start 2024-04-15T15:24:48.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72bca00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72b80725f0 0x7f72b80ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72bca00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f72b80ce9e0 0x7f72b806df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72bca00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b80cd2d0 con 0x7f72b81461c0 2024-04-15T15:24:48.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72bca00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f72b80cf1d0 con 0x7f72b80725f0 2024-04-15T15:24:48.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72bca00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f72b80cd080 con 0x7f72b80ce9e0 2024-04-15T15:24:48.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72b5e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72b80725f0 0x7f72b80ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72b6800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f72b81461c0 0x7f72b8071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56660/0 (socket says 172.21.15.150:56660) 2024-04-15T15:24:48.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72b6800640 1 -- 172.21.15.150:0/3399570638 learned_addr learned my addr 172.21.15.150:0/3399570638 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:48.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.742+0000 7f72b7200640 1 --2- 172.21.15.150:0/3399570638 >> v2:172.21.15.187:3300/0 conn(0x7f72b80ce9e0 0x7f72b806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.743+0000 7f72b5400640 1 -- 172.21.15.150:0/3399570638 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130120016 0 0) 0x7f72b80cd2d0 con 0x7f72b81461c0 2024-04-15T15:24:48.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.743+0000 7f72b5e00640 1 -- 172.21.15.150:0/3399570638 >> v2:172.21.15.187:3300/0 conn(0x7f72b80ce9e0 msgr2=0x7f72b806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:48.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.743+0000 7f72b5e00640 1 --2- 172.21.15.150:0/3399570638 >> v2:172.21.15.187:3300/0 conn(0x7f72b80ce9e0 0x7f72b806df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.743+0000 7f72b5e00640 1 -- 172.21.15.150:0/3399570638 >> v1:172.21.15.150:6789/0 conn(0x7f72b81461c0 legacy=0x7f72b8071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.743+0000 7f72b5e00640 1 -- 172.21.15.150:0/3399570638 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b80cd5b0 con 0x7f72b80725f0 2024-04-15T15:24:48.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.744+0000 7f72b5e00640 1 --2- 172.21.15.150:0/3399570638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80725f0 0x7f72b80ce4a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f72b80b60e0 tx=0x7f72a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ace439b4c71ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:48.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.744+0000 7f72b5400640 1 -- 172.21.15.150:0/3399570638 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4035020 con 0x7f72b80725f0 2024-04-15T15:24:48.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.744+0000 7f72b5400640 1 -- 172.21.15.150:0/3399570638 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a4031830 con 0x7f72b80725f0 2024-04-15T15:24:48.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72b5400640 1 -- 172.21.15.150:0/3399570638 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4031c00 con 0x7f72b80725f0 2024-04-15T15:24:48.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72bca00640 1 -- 172.21.15.150:0/3399570638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80725f0 msgr2=0x7f72b80ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72bca00640 1 --2- 172.21.15.150:0/3399570638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80725f0 0x7f72b80ce4a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f72b80b60e0 tx=0x7f72a402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72bca00640 1 -- 172.21.15.150:0/3399570638 shutdown_connections 2024-04-15T15:24:48.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72bca00640 1 --2- 172.21.15.150:0/3399570638 >> v2:172.21.15.187:3300/0 conn(0x7f72b80ce9e0 0x7f72b806df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72bca00640 1 --2- 172.21.15.150:0/3399570638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80725f0 0x7f72b80ce4a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72bca00640 1 -- 172.21.15.150:0/3399570638 >> 172.21.15.150:0/3399570638 conn(0x7f72b806b930 msgr2=0x7f72b806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:48.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72bca00640 1 -- 172.21.15.150:0/3399570638 shutdown_connections 2024-04-15T15:24:48.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.745+0000 7f72bca00640 1 -- 172.21.15.150:0/3399570638 wait complete. 2024-04-15T15:24:48.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.747+0000 7f72bca00640 1 Processor -- start 2024-04-15T15:24:48.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.747+0000 7f72bca00640 1 -- start start 2024-04-15T15:24:48.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.748+0000 7f72bca00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b80725f0 0x7f72b813cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.748+0000 7f72bca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80ce9e0 0x7f72b8157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.748+0000 7f72bca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72b81461c0 0x7f72b8158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.748+0000 7f72bca00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f72b80cee30 con 0x7f72b80ce9e0 2024-04-15T15:24:48.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.748+0000 7f72bca00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72b80cdb70 con 0x7f72b80725f0 2024-04-15T15:24:48.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.748+0000 7f72bca00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f72b80cefb0 con 0x7f72b81461c0 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.748+0000 7f72b5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80ce9e0 0x7f72b8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80ce9e0 0x7f72b8157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33558/0 (socket says 172.21.15.150:33558) 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b5e00640 1 -- 172.21.15.150:0/2002845658 learned_addr learned my addr 172.21.15.150:0/2002845658 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b7200640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72b81461c0 0x7f72b8158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b6800640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b80725f0 0x7f72b813cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b5e00640 1 -- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72b81461c0 msgr2=0x7f72b8158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b5e00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72b81461c0 0x7f72b8158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b5e00640 1 -- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b80725f0 msgr2=0x7f72b813cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b5e00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b80725f0 0x7f72b813cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b5e00640 1 -- 172.21.15.150:0/2002845658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b815a2c0 con 0x7f72b80ce9e0 2024-04-15T15:24:48.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b6800640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b80725f0 0x7f72b813cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:24:48.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.749+0000 7f72b5e00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80ce9e0 0x7f72b8157770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009890 tx=0x7f72a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:48.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.750+0000 7f72aea00640 1 -- 172.21.15.150:0/2002845658 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4041070 con 0x7f72b80ce9e0 2024-04-15T15:24:48.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.750+0000 7f72aea00640 1 -- 172.21.15.150:0/2002845658 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a403aa30 con 0x7f72b80ce9e0 2024-04-15T15:24:48.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.750+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b8157cb0 con 0x7f72b80ce9e0 2024-04-15T15:24:48.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.750+0000 7f72aea00640 1 -- 172.21.15.150:0/2002845658 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4035040 con 0x7f72b80ce9e0 2024-04-15T15:24:48.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.750+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b815a840 con 0x7f72b80ce9e0 2024-04-15T15:24:48.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.751+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b80755c0 con 0x7f72b80ce9e0 2024-04-15T15:24:48.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.752+0000 7f72aea00640 1 -- 172.21.15.150:0/2002845658 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72a4038020 con 0x7f72b80ce9e0 2024-04-15T15:24:48.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.753+0000 7f72aea00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f729806c8f0 0x7f729806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:48.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.753+0000 7f72aea00640 1 -- 172.21.15.150:0/2002845658 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f72a40ad660 con 0x7f72b80ce9e0 2024-04-15T15:24:48.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.753+0000 7f72b6800640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f729806c8f0 0x7f729806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:48.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.754+0000 7f72b6800640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f729806c8f0 0x7f729806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72a8000f60 tx=0x7f72a8007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:48.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.758+0000 7f72aea00640 1 -- 172.21.15.150:0/2002845658 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72a407b940 con 0x7f72b80ce9e0 2024-04-15T15:24:48.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.958+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f72b8157ee0 con 0x7f72b80ce9e0 2024-04-15T15:24:48.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.959+0000 7f72aea00640 1 -- 172.21.15.150:0/2002845658 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f72a407b240 con 0x7f72b80ce9e0 2024-04-15T15:24:48.961 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:48.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.961+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f729806c8f0 msgr2=0x7f729806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.961+0000 7f72bca00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f729806c8f0 0x7f729806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72a8000f60 tx=0x7f72a8007ca0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.961+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80ce9e0 msgr2=0x7f72b8157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:48.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.961+0000 7f72bca00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80ce9e0 0x7f72b8157770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009890 tx=0x7f72a40046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.962+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 shutdown_connections 2024-04-15T15:24:48.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.962+0000 7f72bca00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f729806c8f0 0x7f729806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.962+0000 7f72bca00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72b81461c0 0x7f72b8158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.962+0000 7f72bca00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b80ce9e0 0x7f72b8157770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.962+0000 7f72bca00640 1 --2- 172.21.15.150:0/2002845658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b80725f0 0x7f72b813cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:48.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.962+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 >> 172.21.15.150:0/2002845658 conn(0x7f72b806b930 msgr2=0x7f72b8143e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:48.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.962+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 shutdown_connections 2024-04-15T15:24:48.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:48.962+0000 7f72bca00640 1 -- 172.21.15.150:0/2002845658 wait complete. 2024-04-15T15:24:48.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:24:49.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.190+0000 7f4c2ce00640 1 Processor -- start 2024-04-15T15:24:49.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.190+0000 7f4c2ce00640 1 -- start start 2024-04-15T15:24:49.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.192+0000 7f4c2ce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c28162b30 0x7f4c281697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.192+0000 7f4c2ce00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c280b3e90 con 0x7f4c28148050 2024-04-15T15:24:49.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.192+0000 7f4c2ce00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c280afba0 con 0x7f4c280ae790 2024-04-15T15:24:49.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.192+0000 7f4c2ce00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4c280aed00 con 0x7f4c28162b30 2024-04-15T15:24:49.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.192+0000 7f4c27400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f4c28148050 0x7f4c280b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56662/0 (socket says 172.21.15.150:56662) 2024-04-15T15:24:49.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c27400640 1 -- 172.21.15.150:0/3579658591 learned_addr learned my addr 172.21.15.150:0/3579658591 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:49.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c27e00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c28162b30 0x7f4c281697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3976019621 0 0) 0x7f4c280b3e90 con 0x7f4c28148050 2024-04-15T15:24:49.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c0c003520 con 0x7f4c28148050 2024-04-15T15:24:49.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3280383330 0 0) 0x7f4c0c003520 con 0x7f4c28148050 2024-04-15T15:24:49.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c28162b30 msgr2=0x7f4c281697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:49.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c28162b30 0x7f4c281697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 >> v1:172.21.15.187:6789/0 conn(0x7f4c280ae790 legacy=0x7f4c28162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c280aefe0 con 0x7f4c28148050 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f4c18001670 con 0x7f4c28148050 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 >> v1:172.21.15.150:6789/0 conn(0x7f4c28148050 legacy=0x7f4c280b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.193+0000 7f4c26a00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c28162b30 0x7f4c0c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c26a00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c0c003f60 0x7f4c0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c26a00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c0c004d40 0x7f4c0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4c280aed00 con 0x7f4c0c003f60 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c00baa0 con 0x7f4c0c004d40 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0c00bca0 con 0x7f4c28162b30 2024-04-15T15:24:49.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c1fe00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c0c003f60 0x7f4c0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c27e00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c0c004d40 0x7f4c0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c1fe00640 1 -- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c28162b30 msgr2=0x7f4c0c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:49.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c1fe00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c28162b30 0x7f4c0c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c1fe00640 1 -- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c0c004d40 msgr2=0x7f4c0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:49.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c1fe00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c0c004d40 0x7f4c0c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.194+0000 7f4c1fe00640 1 -- 172.21.15.150:0/3579658591 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c003520 con 0x7f4c0c003f60 2024-04-15T15:24:49.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.195+0000 7f4c1fe00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c0c003f60 0x7f4c0c004800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c10001750 tx=0x7f4c0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6017f5ca0d5316 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:49.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10001370 con 0x7f4c0c003f60 2024-04-15T15:24:49.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c100052d0 con 0x7f4c0c003f60 2024-04-15T15:24:49.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c26a00640 1 -- 172.21.15.150:0/3579658591 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10003830 con 0x7f4c0c003f60 2024-04-15T15:24:49.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c2ce00640 1 -- 172.21.15.150:0/3579658591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c0c003f60 msgr2=0x7f4c0c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c2ce00640 1 --2- 172.21.15.150:0/3579658591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c0c003f60 0x7f4c0c004800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c10001750 tx=0x7f4c0c005830 comp rx=0 tx=0).stop 2024-04-15T15:24:49.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c27e00640 1 -- 172.21.15.150:0/3579658591 reap_dead start 2024-04-15T15:24:49.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c2ce00640 1 -- 172.21.15.150:0/3579658591 shutdown_connections 2024-04-15T15:24:49.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c2ce00640 1 -- 172.21.15.150:0/3579658591 >> 172.21.15.150:0/3579658591 conn(0x7f4c280b23a0 msgr2=0x7f4c280b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:49.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c2ce00640 1 -- 172.21.15.150:0/3579658591 shutdown_connections 2024-04-15T15:24:49.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.196+0000 7f4c2ce00640 1 -- 172.21.15.150:0/3579658591 wait complete. 2024-04-15T15:24:49.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.198+0000 7f4c2ce00640 1 Processor -- start 2024-04-15T15:24:49.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.198+0000 7f4c2ce00640 1 -- start start 2024-04-15T15:24:49.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c2ce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c0c003f60 0x7f4c2815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c2ce00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c2815eef0 0x7f4c2815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c2ce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2815f8b0 0x7f4c28160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c2ce00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4c28148430 con 0x7f4c2815f8b0 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c2ce00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c280afe40 con 0x7f4c2815eef0 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c2ce00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4c281485b0 con 0x7f4c0c003f60 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2815f8b0 0x7f4c28160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2815f8b0 0x7f4c28160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33578/0 (socket says 172.21.15.150:33578) 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.199+0000 7f4c27e00640 1 -- 172.21.15.150:0/2004697047 learned_addr learned my addr 172.21.15.150:0/2004697047 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c27400640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c0c003f60 0x7f4c2815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c1fe00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c2815eef0 0x7f4c2815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c27e00640 1 -- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c0c003f60 msgr2=0x7f4c2815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c27e00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c0c003f60 0x7f4c2815ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c27e00640 1 -- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c2815eef0 msgr2=0x7f4c2815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c27e00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c2815eef0 0x7f4c2815f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c27e00640 1 -- 172.21.15.150:0/2004697047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c280cdc10 con 0x7f4c2815f8b0 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c27e00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2815f8b0 0x7f4c28160430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4c2806fed0 tx=0x7f4c2000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.200+0000 7f4c1f400640 1 -- 172.21.15.150:0/2004697047 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000fe90 con 0x7f4c2815f8b0 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.201+0000 7f4c1f400640 1 -- 172.21.15.150:0/2004697047 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20003990 con 0x7f4c2815f8b0 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.201+0000 7f4c1f400640 1 -- 172.21.15.150:0/2004697047 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20003d60 con 0x7f4c2815f8b0 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.201+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c280cdeb0 con 0x7f4c2815f8b0 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.201+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c280ce470 con 0x7f4c2815f8b0 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.202+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c280755c0 con 0x7f4c2815f8b0 2024-04-15T15:24:49.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.203+0000 7f4c1f400640 1 -- 172.21.15.150:0/2004697047 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c2001e020 con 0x7f4c2815f8b0 2024-04-15T15:24:49.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.203+0000 7f4c1f400640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4c0406cac0 0x7f4c0406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.204+0000 7f4c1f400640 1 -- 172.21.15.150:0/2004697047 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f4c2008bfe0 con 0x7f4c2815f8b0 2024-04-15T15:24:49.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.204+0000 7f4c27400640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4c0406cac0 0x7f4c0406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.205+0000 7f4c27400640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4c0406cac0 0x7f4c0406ef80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4c18001860 tx=0x7f4c18001610 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:49.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.208+0000 7f4c1f400640 1 -- 172.21.15.150:0/2004697047 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c20012070 con 0x7f4c2815f8b0 2024-04-15T15:24:49.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.419+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c280b3240 con 0x7f4c2815f8b0 2024-04-15T15:24:49.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.420+0000 7f4c1f400640 1 -- 172.21.15.150:0/2004697047 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f4c2005a350 con 0x7f4c2815f8b0 2024-04-15T15:24:49.422 INFO:teuthology.orchestra.run.smithi150.stderr:listed 0 entries 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4c0406cac0 msgr2=0x7f4c0406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4c0406cac0 0x7f4c0406ef80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4c18001860 tx=0x7f4c18001610 comp rx=0 tx=0).stop 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2815f8b0 msgr2=0x7f4c28160430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2815f8b0 0x7f4c28160430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4c2806fed0 tx=0x7f4c2000f4a0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 shutdown_connections 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2815f8b0 0x7f4c28160430 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c2815eef0 0x7f4c2815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4c0406cac0 0x7f4c0406ef80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 --2- 172.21.15.150:0/2004697047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c0c003f60 0x7f4c2815ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 >> 172.21.15.150:0/2004697047 conn(0x7f4c280b23a0 msgr2=0x7f4c2806ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:49.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 shutdown_connections 2024-04-15T15:24:49.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.423+0000 7f4c2ce00640 1 -- 172.21.15.150:0/2004697047 wait complete. 2024-04-15T15:24:49.441 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-15T15:24:49.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:49.441 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-15T15:24:49.552 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:24:49.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:24:49.552 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:24:49.603 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-15T15:24:49.604 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:24:49.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.815+0000 7fdfa1000640 1 Processor -- start 2024-04-15T15:24:49.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.815+0000 7fdfa1000640 1 -- start start 2024-04-15T15:24:49.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.816+0000 7fdfa1000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdf9c072660 0x7fdf9c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.816+0000 7fdfa1000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c15cbe0 0x7fdf9c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.816+0000 7fdfa1000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c15dbb0 con 0x7fdf9c0cee00 2024-04-15T15:24:49.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.816+0000 7fdfa1000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c06f4f0 con 0x7fdf9c072660 2024-04-15T15:24:49.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.816+0000 7fdfa1000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c15d8d0 con 0x7fdf9c15cbe0 2024-04-15T15:24:49.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.817+0000 7fdf9b400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fdf9c0cee00 0x7fdf9c071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56670/0 (socket says 172.21.15.150:56670) 2024-04-15T15:24:49.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.817+0000 7fdf9b400640 1 -- 172.21.15.150:0/1445598301 learned_addr learned my addr 172.21.15.150:0/1445598301 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.817+0000 7fdf93e00640 1 --2- 172.21.15.150:0/1445598301 >> v2:172.21.15.187:3300/0 conn(0x7fdf9c072660 0x7fdf9c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.817+0000 7fdf9be00640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c15cbe0 0x7fdf9c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.817+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3332500684 0 0) 0x7fdf9c15dbb0 con 0x7fdf9c0cee00 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.817+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf80003520 con 0x7fdf9c0cee00 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4089060669 0 0) 0x7fdf80003520 con 0x7fdf9c0cee00 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c15cbe0 msgr2=0x7fdf9c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c15cbe0 0x7fdf9c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 >> v2:172.21.15.187:3300/0 conn(0x7fdf9c072660 msgr2=0x7fdf9c06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 --2- 172.21.15.150:0/1445598301 >> v2:172.21.15.187:3300/0 conn(0x7fdf9c072660 0x7fdf9c06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c06dde0 con 0x7fdf9c0cee00 2024-04-15T15:24:49.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fdf8c001670 con 0x7fdf9c0cee00 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 >> v1:172.21.15.150:6789/0 conn(0x7fdf9c0cee00 legacy=0x7fdf9c071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf80004800 0x7fdf80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c15cbe0 0x7fdf800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c15d8d0 con 0x7fdf800039b0 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8000b9c0 con 0x7fdf80004800 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8000bb40 con 0x7fdf9c15cbe0 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9b400640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9b400640 1 -- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c15cbe0 msgr2=0x7fdf800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9b400640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c15cbe0 0x7fdf800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf93e00640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf80004800 0x7fdf80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.818+0000 7fdf9b400640 1 -- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf80004800 msgr2=0x7fdf80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:49.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.819+0000 7fdf9b400640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf80004800 0x7fdf80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.819+0000 7fdf9b400640 1 -- 172.21.15.150:0/1445598301 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf80003520 con 0x7fdf800039b0 2024-04-15T15:24:49.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.819+0000 7fdf93e00640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf80004800 0x7fdf80004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:49.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.820+0000 7fdf9b400640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf80003dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c290 tx=0x7fdf80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20b1354a21a97be server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:49.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.820+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c007d50 con 0x7fdf800039b0 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.820+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c030070 con 0x7fdf800039b0 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.820+0000 7fdf9aa00640 1 -- 172.21.15.150:0/1445598301 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c02cd80 con 0x7fdf800039b0 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.820+0000 7fdfa1000640 1 -- 172.21.15.150:0/1445598301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 msgr2=0x7fdf80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.820+0000 7fdfa1000640 1 --2- 172.21.15.150:0/1445598301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf80003dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c290 tx=0x7fdf80005830 comp rx=0 tx=0).stop 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.820+0000 7fdf9be00640 1 -- 172.21.15.150:0/1445598301 reap_dead start 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.821+0000 7fdfa1000640 1 -- 172.21.15.150:0/1445598301 shutdown_connections 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.821+0000 7fdfa1000640 1 -- 172.21.15.150:0/1445598301 >> 172.21.15.150:0/1445598301 conn(0x7fdf9c06b930 msgr2=0x7fdf9c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.821+0000 7fdfa1000640 1 -- 172.21.15.150:0/1445598301 shutdown_connections 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.821+0000 7fdfa1000640 1 -- 172.21.15.150:0/1445598301 wait complete. 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.822+0000 7fdfa1000640 1 Processor -- start 2024-04-15T15:24:49.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.823+0000 7fdfa1000640 1 -- start start 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.823+0000 7fdfa1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf9c140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.823+0000 7fdfa1000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c14f870 0x7fdf9c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.823+0000 7fdfa1000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf9c14fcd0 0x7fdf9c1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.823+0000 7fdfa1000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c15d070 con 0x7fdf800039b0 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.823+0000 7fdfa1000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c06e3a0 con 0x7fdf9c14fcd0 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.823+0000 7fdfa1000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c15d1f0 con 0x7fdf9c14f870 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf9c140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf9c140f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33602/0 (socket says 172.21.15.150:33602) 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9b400640 1 -- 172.21.15.150:0/3131172230 learned_addr learned my addr 172.21.15.150:0/3131172230 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:49.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf93e00640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c14f870 0x7fdf9c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9be00640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf9c14fcd0 0x7fdf9c1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9b400640 1 -- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c14f870 msgr2=0x7fdf9c151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9b400640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c14f870 0x7fdf9c151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9b400640 1 -- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf9c14fcd0 msgr2=0x7fdf9c1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9b400640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf9c14fcd0 0x7fdf9c1519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9b400640 1 -- 172.21.15.150:0/3131172230 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c1522f0 con 0x7fdf800039b0 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.824+0000 7fdf9be00640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf9c14fcd0 0x7fdf9c1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.825+0000 7fdf9b400640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf9c140f10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c380 tx=0x7fdf9c0a94a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.825+0000 7fdf93400640 1 -- 172.21.15.150:0/3131172230 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c046070 con 0x7fdf800039b0 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.825+0000 7fdf93400640 1 -- 172.21.15.150:0/3131172230 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c004bf0 con 0x7fdf800039b0 2024-04-15T15:24:49.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.825+0000 7fdf93400640 1 -- 172.21.15.150:0/3131172230 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c007d50 con 0x7fdf800039b0 2024-04-15T15:24:49.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.825+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c1525d0 con 0x7fdf800039b0 2024-04-15T15:24:49.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.825+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c152b90 con 0x7fdf800039b0 2024-04-15T15:24:49.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.826+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c0755c0 con 0x7fdf800039b0 2024-04-15T15:24:49.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.827+0000 7fdf93400640 1 -- 172.21.15.150:0/3131172230 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf8c007070 con 0x7fdf800039b0 2024-04-15T15:24:49.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.828+0000 7fdf93400640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf7806c8f0 0x7fdf7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:49.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.828+0000 7fdf93400640 1 -- 172.21.15.150:0/3131172230 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fdf8c0acf20 con 0x7fdf800039b0 2024-04-15T15:24:49.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.829+0000 7fdf93e00640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf7806c8f0 0x7fdf7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:49.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.830+0000 7fdf93e00640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf7806c8f0 0x7fdf7806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c15a330 tx=0x7fdf84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:49.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:49.833+0000 7fdf93400640 1 -- 172.21.15.150:0/3131172230 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf8c07b230 con 0x7fdf800039b0 2024-04-15T15:24:50.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.071+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf9c06f800 con 0x7fdf800039b0 2024-04-15T15:24:50.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.073+0000 7fdf93400640 1 -- 172.21.15.150:0/3131172230 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7fdf8c07ab30 con 0x7fdf800039b0 2024-04-15T15:24:50.075 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:50.075 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:23:54.683053+0000","modified":"2024-04-15T15:23:57.779875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4250,"qdb_cluster":[4250]},"id":1}]} 2024-04-15T15:24:50.075 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-04-15T15:24:50.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.075+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf7806c8f0 msgr2=0x7fdf7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.075+0000 7fdfa1000640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf7806c8f0 0x7fdf7806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c15a330 tx=0x7fdf84008040 comp rx=0 tx=0).stop 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.075+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 msgr2=0x7fdf9c140f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.075+0000 7fdfa1000640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf9c140f10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c380 tx=0x7fdf9c0a94a0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.076+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 shutdown_connections 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.076+0000 7fdfa1000640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf9c14fcd0 0x7fdf9c1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.076+0000 7fdfa1000640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdf7806c8f0 0x7fdf7806edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.076+0000 7fdfa1000640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf9c14f870 0x7fdf9c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.076+0000 7fdfa1000640 1 --2- 172.21.15.150:0/3131172230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf800039b0 0x7fdf9c140f10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.076+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 >> 172.21.15.150:0/3131172230 conn(0x7fdf9c06b930 msgr2=0x7fdf9c160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.076+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 shutdown_connections 2024-04-15T15:24:50.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.076+0000 7fdfa1000640 1 -- 172.21.15.150:0/3131172230 wait complete. 2024-04-15T15:24:50.096 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:50.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.304+0000 7f59ae600640 1 Processor -- start 2024-04-15T15:24:50.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.304+0000 7f59ae600640 1 -- start start 2024-04-15T15:24:50.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.306+0000 7f59ae600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a8070210 0x7f59a80b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.306+0000 7f59ae600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a806f940 con 0x7f59a80b5020 2024-04-15T15:24:50.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.306+0000 7f59ae600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a806e8a0 con 0x7f59a815b6d0 2024-04-15T15:24:50.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.306+0000 7f59ae600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59a80b5480 con 0x7f59a8070210 2024-04-15T15:24:50.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.306+0000 7f59a7e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f59a80b5020 0x7f59a815bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56682/0 (socket says 172.21.15.150:56682) 2024-04-15T15:24:50.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.306+0000 7f59a7e00640 1 -- 172.21.15.150:0/4121630794 learned_addr learned my addr 172.21.15.150:0/4121630794 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:50.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6a00640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a8070210 0x7f59a80b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3058775127 0 0) 0x7f59a806f940 con 0x7f59a80b5020 2024-04-15T15:24:50.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f598c003520 con 0x7f59a80b5020 2024-04-15T15:24:50.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5312250 0 0) 0x7f598c003520 con 0x7f59a80b5020 2024-04-15T15:24:50.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a8070210 msgr2=0x7f59a80b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a8070210 0x7f59a80b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 >> v1:172.21.15.187:6789/0 conn(0x7f59a815b6d0 legacy=0x7f59a80cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a806eb80 con 0x7f59a80b5020 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f59a00019f0 con 0x7f59a80b5020 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 >> v1:172.21.15.150:6789/0 conn(0x7f59a80b5020 legacy=0x7f59a815bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c0039e0 0x7f598c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a8070210 0x7f598c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c004d40 0x7f598c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f598c00bb50 con 0x7f598c004d40 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f598c00bd00 con 0x7f598c0039e0 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.307+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f598c00bf00 con 0x7f59a8070210 2024-04-15T15:24:50.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.308+0000 7f59a7e00640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c004d40 0x7f598c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.308+0000 7f59a6a00640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a8070210 0x7f598c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.308+0000 7f59a7400640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c0039e0 0x7f598c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.308+0000 7f59a7e00640 1 -- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a8070210 msgr2=0x7f598c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:50.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.308+0000 7f59a7e00640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a8070210 0x7f598c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.308+0000 7f59a7e00640 1 -- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c0039e0 msgr2=0x7f598c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:50.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.308+0000 7f59a7e00640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c0039e0 0x7f598c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.308+0000 7f59a7e00640 1 -- 172.21.15.150:0/4121630794 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598c003520 con 0x7f598c004d40 2024-04-15T15:24:50.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.309+0000 7f59a7e00640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c004d40 0x7f598c0051c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f59a000a350 tx=0x7f598c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356b8955785d2d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:50.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.310+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a000a390 con 0x7f598c004d40 2024-04-15T15:24:50.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.310+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a0036630 con 0x7f598c004d40 2024-04-15T15:24:50.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.310+0000 7f59a6000640 1 -- 172.21.15.150:0/4121630794 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0036a20 con 0x7f598c004d40 2024-04-15T15:24:50.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.310+0000 7f59ae600640 1 -- 172.21.15.150:0/4121630794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c004d40 msgr2=0x7f598c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.310+0000 7f59ae600640 1 --2- 172.21.15.150:0/4121630794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c004d40 0x7f598c0051c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f59a000a350 tx=0x7f598c005830 comp rx=0 tx=0).stop 2024-04-15T15:24:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.310+0000 7f59a7e00640 1 -- 172.21.15.150:0/4121630794 reap_dead start 2024-04-15T15:24:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.310+0000 7f59ae600640 1 -- 172.21.15.150:0/4121630794 shutdown_connections 2024-04-15T15:24:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.310+0000 7f59ae600640 1 -- 172.21.15.150:0/4121630794 >> 172.21.15.150:0/4121630794 conn(0x7f59a80b27b0 msgr2=0x7f59a80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.311+0000 7f59ae600640 1 -- 172.21.15.150:0/4121630794 shutdown_connections 2024-04-15T15:24:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.311+0000 7f59ae600640 1 -- 172.21.15.150:0/4121630794 wait complete. 2024-04-15T15:24:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.313+0000 7f59ae600640 1 Processor -- start 2024-04-15T15:24:50.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.313+0000 7f59ae600640 1 -- start start 2024-04-15T15:24:50.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59ae600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c004d40 0x7f59a8154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59ae600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a80cb4f0 0x7f59a80b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59ae600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a80b1d30 0x7f59a80c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59ae600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59a806f420 con 0x7f59a80cb4f0 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59ae600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59a806f140 con 0x7f598c004d40 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59ae600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59a80ce420 con 0x7f59a80b1d30 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59a6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a80cb4f0 0x7f59a80b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59a6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a80cb4f0 0x7f59a80b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33618/0 (socket says 172.21.15.150:33618) 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.314+0000 7f59a6a00640 1 -- 172.21.15.150:0/1633799064 learned_addr learned my addr 172.21.15.150:0/1633799064 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a7e00640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a80b1d30 0x7f59a80c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a7400640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c004d40 0x7f59a8154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a6a00640 1 -- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a80b1d30 msgr2=0x7f59a80c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a6a00640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a80b1d30 0x7f59a80c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a6a00640 1 -- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c004d40 msgr2=0x7f59a8154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a6a00640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c004d40 0x7f59a8154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a6a00640 1 -- 172.21.15.150:0/1633799064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a81f5b80 con 0x7f59a80cb4f0 2024-04-15T15:24:50.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a7e00640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a80b1d30 0x7f59a80c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:50.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a7400640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c004d40 0x7f59a8154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:50.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f59a6a00640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a80cb4f0 0x7f59a80b17f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5994011600 tx=0x7f5994011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.315+0000 7f599f400640 1 -- 172.21.15.150:0/1633799064 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599400f920 con 0x7f59a80cb4f0 2024-04-15T15:24:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.316+0000 7f599f400640 1 -- 172.21.15.150:0/1633799064 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599400fac0 con 0x7f59a80cb4f0 2024-04-15T15:24:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.316+0000 7f599f400640 1 -- 172.21.15.150:0/1633799064 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599400fe70 con 0x7f59a80cb4f0 2024-04-15T15:24:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.316+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a80ca490 con 0x7f59a80cb4f0 2024-04-15T15:24:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.316+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a80ca730 con 0x7f59a80cb4f0 2024-04-15T15:24:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.317+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a80cf510 con 0x7f59a80cb4f0 2024-04-15T15:24:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.318+0000 7f599f400640 1 -- 172.21.15.150:0/1633799064 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5994015460 con 0x7f59a80cb4f0 2024-04-15T15:24:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.319+0000 7f599f400640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f598806c8f0 0x7f598806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.319+0000 7f599f400640 1 -- 172.21.15.150:0/1633799064 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f599408c2a0 con 0x7f59a80cb4f0 2024-04-15T15:24:50.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.319+0000 7f59a7400640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f598806c8f0 0x7f598806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.321+0000 7f59a7400640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f598806c8f0 0x7f598806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5998000c00 tx=0x7f5998009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:50.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.325+0000 7f599f400640 1 -- 172.21.15.150:0/1633799064 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59a80cf510 con 0x7f59a80cb4f0 2024-04-15T15:24:50.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.533+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59a8068c80 con 0x7f59a80cb4f0 2024-04-15T15:24:50.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.535+0000 7f599f400640 1 -- 172.21.15.150:0/1633799064 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16997 (secure 0 0 0) 0x7f5994019030 con 0x7f59a80cb4f0 2024-04-15T15:24:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:50.539 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:36.500392+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f598806c8f0 msgr2=0x7f598806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f598806c8f0 0x7f598806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5998000c00 tx=0x7f5998009040 comp rx=0 tx=0).stop 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a80cb4f0 msgr2=0x7f59a80b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a80cb4f0 0x7f59a80b17f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5994011600 tx=0x7f5994011ac0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 shutdown_connections 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f598806c8f0 0x7f598806edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59a80b1d30 0x7f59a80c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a80cb4f0 0x7f59a80b17f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 --2- 172.21.15.150:0/1633799064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f598c004d40 0x7f59a8154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 >> 172.21.15.150:0/1633799064 conn(0x7f59a80b27b0 msgr2=0x7f59a80b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:50.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 shutdown_connections 2024-04-15T15:24:50.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.538+0000 7f59ae600640 1 -- 172.21.15.150:0/1633799064 wait complete. 2024-04-15T15:24:50.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:23:53.008505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:23:53.658759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:24:50.559 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:24:50.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.775+0000 7f0db0000640 1 Processor -- start 2024-04-15T15:24:50.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.775+0000 7f0db0000640 1 -- start start 2024-04-15T15:24:50.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.776+0000 7f0db0000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0da8162f90 0x7f0da80ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0db0000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0da8072860 0x7f0da8166e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0db0000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da80cec10 con 0x7f0da80afbc0 2024-04-15T15:24:50.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0db0000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0da80b46c0 con 0x7f0da8072860 2024-04-15T15:24:50.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0db0000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0da806fe90 con 0x7f0da8162f90 2024-04-15T15:24:50.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0dacc00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0da80afbc0 0x7f0da8072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56696/0 (socket says 172.21.15.150:56696) 2024-04-15T15:24:50.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0dacc00640 1 -- 172.21.15.150:0/2200661117 learned_addr learned my addr 172.21.15.150:0/2200661117 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:50.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0dae000640 1 --2- 172.21.15.150:0/2200661117 >> v2:172.21.15.150:3300/0 conn(0x7f0da8072860 0x7f0da8166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0dad600640 1 --2- 172.21.15.150:0/2200661117 >> v2:172.21.15.187:3300/0 conn(0x7f0da8162f90 0x7f0da80ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.777+0000 7f0da7e00640 1 -- 172.21.15.150:0/2200661117 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2174090544 0 0) 0x7f0da80cec10 con 0x7f0da80afbc0 2024-04-15T15:24:50.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.778+0000 7f0da7e00640 1 -- 172.21.15.150:0/2200661117 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d90003520 con 0x7f0da80afbc0 2024-04-15T15:24:50.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.778+0000 7f0dae000640 1 -- 172.21.15.150:0/2200661117 >> v2:172.21.15.187:3300/0 conn(0x7f0da8162f90 msgr2=0x7f0da80ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:50.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.778+0000 7f0dae000640 1 --2- 172.21.15.150:0/2200661117 >> v2:172.21.15.187:3300/0 conn(0x7f0da8162f90 0x7f0da80ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.778+0000 7f0dae000640 1 -- 172.21.15.150:0/2200661117 >> v1:172.21.15.150:6789/0 conn(0x7f0da80afbc0 legacy=0x7f0da8072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.778+0000 7f0dae000640 1 -- 172.21.15.150:0/2200661117 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da80b3ab0 con 0x7f0da8072860 2024-04-15T15:24:50.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.779+0000 7f0dae000640 1 --2- 172.21.15.150:0/2200661117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da8166e30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0da8072e80 tx=0x7f0da002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25005da0e204db25 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:50.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.779+0000 7f0da7e00640 1 -- 172.21.15.150:0/2200661117 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0039070 con 0x7f0da8072860 2024-04-15T15:24:50.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.779+0000 7f0da7e00640 1 -- 172.21.15.150:0/2200661117 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da002cc80 con 0x7f0da8072860 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.779+0000 7f0da7e00640 1 -- 172.21.15.150:0/2200661117 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0034640 con 0x7f0da8072860 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.780+0000 7f0db0000640 1 -- 172.21.15.150:0/2200661117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 msgr2=0x7f0da8166e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.780+0000 7f0db0000640 1 --2- 172.21.15.150:0/2200661117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da8166e30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0da8072e80 tx=0x7f0da002bed0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.780+0000 7f0db0000640 1 -- 172.21.15.150:0/2200661117 shutdown_connections 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.780+0000 7f0db0000640 1 --2- 172.21.15.150:0/2200661117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da8166e30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.780+0000 7f0db0000640 1 --2- 172.21.15.150:0/2200661117 >> v2:172.21.15.187:3300/0 conn(0x7f0da8162f90 0x7f0da80ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.780+0000 7f0db0000640 1 -- 172.21.15.150:0/2200661117 >> 172.21.15.150:0/2200661117 conn(0x7f0da80b2460 msgr2=0x7f0da80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.780+0000 7f0db0000640 1 -- 172.21.15.150:0/2200661117 shutdown_connections 2024-04-15T15:24:50.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.780+0000 7f0db0000640 1 -- 172.21.15.150:0/2200661117 wait complete. 2024-04-15T15:24:50.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.782+0000 7f0db0000640 1 Processor -- start 2024-04-15T15:24:50.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.782+0000 7f0db0000640 1 -- start start 2024-04-15T15:24:50.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0db0000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da80ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0db0000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da80afbc0 0x7f0da80cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0db0000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da8162f90 0x7f0da80cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0db0000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0da80b4350 con 0x7f0da8072860 2024-04-15T15:24:50.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0db0000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0da80b4070 con 0x7f0da8162f90 2024-04-15T15:24:50.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0db0000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0da8163370 con 0x7f0da80afbc0 2024-04-15T15:24:50.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0dad600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da80ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0dad600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da80ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33636/0 (socket says 172.21.15.150:33636) 2024-04-15T15:24:50.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0dad600640 1 -- 172.21.15.150:0/1662994606 learned_addr learned my addr 172.21.15.150:0/1662994606 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:50.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0dad600640 1 -- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da80afbc0 msgr2=0x7f0da80cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0dad600640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da80afbc0 0x7f0da80cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0dad600640 1 -- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da8162f90 msgr2=0x7f0da80cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0dad600640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da8162f90 0x7f0da80cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.783+0000 7f0dad600640 1 -- 172.21.15.150:0/1662994606 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da80ae890 con 0x7f0da8072860 2024-04-15T15:24:50.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.784+0000 7f0dad600640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da80ae1d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d620 tx=0x7f0d9c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:50.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.784+0000 7f0da5600640 1 -- 172.21.15.150:0/1662994606 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00b620 con 0x7f0da8072860 2024-04-15T15:24:50.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.784+0000 7f0da5600640 1 -- 172.21.15.150:0/1662994606 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c019070 con 0x7f0da8072860 2024-04-15T15:24:50.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.784+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da80aeb70 con 0x7f0da8072860 2024-04-15T15:24:50.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.784+0000 7f0da5600640 1 -- 172.21.15.150:0/1662994606 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00ba00 con 0x7f0da8072860 2024-04-15T15:24:50.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.784+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da80af130 con 0x7f0da8072860 2024-04-15T15:24:50.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.785+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da80af330 con 0x7f0da8072860 2024-04-15T15:24:50.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.786+0000 7f0da5600640 1 -- 172.21.15.150:0/1662994606 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d9c00bba0 con 0x7f0da8072860 2024-04-15T15:24:50.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.787+0000 7f0da5600640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8806c8f0 0x7f0d8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:50.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.787+0000 7f0da5600640 1 -- 172.21.15.150:0/1662994606 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f0d9c08c000 con 0x7f0da8072860 2024-04-15T15:24:50.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.788+0000 7f0dacc00640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8806c8f0 0x7f0d8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:50.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.789+0000 7f0dacc00640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8806c8f0 0x7f0d8806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0d94000f60 tx=0x7f0d940078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:50.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.792+0000 7f0da5600640 1 -- 172.21.15.150:0/1662994606 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d9c05a380 con 0x7f0da8072860 2024-04-15T15:24:50.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.992+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0da8068c80 con 0x7f0da8072860 2024-04-15T15:24:50.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.993+0000 7f0da5600640 1 -- 172.21.15.150:0/1662994606 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f0d9c059c80 con 0x7f0da8072860 2024-04-15T15:24:50.995 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:24:50.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8806c8f0 msgr2=0x7f0d8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8806c8f0 0x7f0d8806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0d94000f60 tx=0x7f0d940078f0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 msgr2=0x7f0da80ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:50.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da80ae1d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d620 tx=0x7f0d9c00dae0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 shutdown_connections 2024-04-15T15:24:50.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0da8162f90 0x7f0da80cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0d8806c8f0 0x7f0d8806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0da80afbc0 0x7f0da80cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 --2- 172.21.15.150:0/1662994606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da8072860 0x7f0da80ae1d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:50.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 >> 172.21.15.150:0/1662994606 conn(0x7f0da80b2460 msgr2=0x7f0da80ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:50.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 shutdown_connections 2024-04-15T15:24:51.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:50.995+0000 7f0db0000640 1 -- 172.21.15.150:0/1662994606 wait complete. 2024-04-15T15:24:51.017 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:24:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.231+0000 7ff699e00640 1 Processor -- start 2024-04-15T15:24:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.231+0000 7ff699e00640 1 -- start start 2024-04-15T15:24:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff699e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff69415a440 0x7ff69415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff699e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69415d520 con 0x7ff694072150 2024-04-15T15:24:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff699e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69415d800 con 0x7ff69406fc80 2024-04-15T15:24:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff699e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff69406e280 con 0x7ff69415a440 2024-04-15T15:24:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff693400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7ff694072150 0x7ff69406e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56700/0 (socket says 172.21.15.150:56700) 2024-04-15T15:24:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff693400640 1 -- 172.21.15.150:0/4157826036 learned_addr learned my addr 172.21.15.150:0/4157826036 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:51.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393406514 0 0) 0x7ff69415d520 con 0x7ff694072150 2024-04-15T15:24:51.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff692a00640 1 --2- 172.21.15.150:0/4157826036 >> v2:172.21.15.187:3300/0 conn(0x7ff69415a440 0x7ff69415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.233+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff678003520 con 0x7ff694072150 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 319463292 0 0) 0x7ff678003520 con 0x7ff694072150 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 >> v2:172.21.15.187:3300/0 conn(0x7ff69415a440 msgr2=0x7ff69415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 --2- 172.21.15.150:0/4157826036 >> v2:172.21.15.187:3300/0 conn(0x7ff69415a440 0x7ff69415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 >> v1:172.21.15.187:6789/0 conn(0x7ff69406fc80 legacy=0x7ff6940cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69415dae0 con 0x7ff694072150 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7ff68c0019f0 con 0x7ff694072150 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 >> v1:172.21.15.150:6789/0 conn(0x7ff694072150 legacy=0x7ff69406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6780039b0 0x7ff678003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff678004800 0x7ff678004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6780051c0 0x7ff67800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff69406e280 con 0x7ff6780051c0 2024-04-15T15:24:51.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff67800c090 con 0x7ff6780039b0 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff67800c290 con 0x7ff678004800 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.234+0000 7ff693400640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6780051c0 0x7ff67800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.235+0000 7ff692a00640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6780039b0 0x7ff678003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.235+0000 7ff692000640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff678004800 0x7ff678004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.235+0000 7ff693400640 1 -- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff678004800 msgr2=0x7ff678004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.235+0000 7ff693400640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff678004800 0x7ff678004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.235+0000 7ff693400640 1 -- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6780039b0 msgr2=0x7ff678003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.235+0000 7ff693400640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6780039b0 0x7ff678003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.235+0000 7ff693400640 1 -- 172.21.15.150:0/4157826036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff678003520 con 0x7ff6780051c0 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.235+0000 7ff692a00640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6780039b0 0x7ff678003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.236+0000 7ff693400640 1 -- 172.21.15.150:0/4157826036 reap_dead start 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.236+0000 7ff693400640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6780051c0 0x7ff67800b9c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00a880 tx=0x7ff678005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d4a7f6c96e5bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.236+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c00a4a0 con 0x7ff6780051c0 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.236+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68c034650 con 0x7ff6780051c0 2024-04-15T15:24:51.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.236+0000 7ff691600640 1 -- 172.21.15.150:0/4157826036 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c034a40 con 0x7ff6780051c0 2024-04-15T15:24:51.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.237+0000 7ff699e00640 1 -- 172.21.15.150:0/4157826036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6780051c0 msgr2=0x7ff67800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.237+0000 7ff699e00640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6780051c0 0x7ff67800b9c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00a880 tx=0x7ff678005830 comp rx=0 tx=0).stop 2024-04-15T15:24:51.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.237+0000 7ff699e00640 1 -- 172.21.15.150:0/4157826036 shutdown_connections 2024-04-15T15:24:51.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.237+0000 7ff699e00640 1 --2- 172.21.15.150:0/4157826036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6780051c0 0x7ff67800b9c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.237+0000 7ff699e00640 1 -- 172.21.15.150:0/4157826036 >> 172.21.15.150:0/4157826036 conn(0x7ff6940b2460 msgr2=0x7ff6940b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:51.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.237+0000 7ff699e00640 1 -- 172.21.15.150:0/4157826036 shutdown_connections 2024-04-15T15:24:51.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.237+0000 7ff699e00640 1 -- 172.21.15.150:0/4157826036 wait complete. 2024-04-15T15:24:51.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.239+0000 7ff699e00640 1 Processor -- start 2024-04-15T15:24:51.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.239+0000 7ff699e00640 1 -- start start 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff699e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6780051c0 0x7ff6940b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff699e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6940af190 0x7ff694158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff699e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940adc00 0x7ff69415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff699e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff69406f510 con 0x7ff6940af190 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff699e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff69415ddc0 con 0x7ff6940adc00 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff699e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff69415cef0 con 0x7ff6780051c0 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff692000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6940af190 0x7ff694158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff692000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6940af190 0x7ff694158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33660/0 (socket says 172.21.15.150:33660) 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.240+0000 7ff692000640 1 -- 172.21.15.150:0/2103827113 learned_addr learned my addr 172.21.15.150:0/2103827113 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff692a00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6780051c0 0x7ff6940b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff692000640 1 -- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6780051c0 msgr2=0x7ff6940b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff692000640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6780051c0 0x7ff6940b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff693400640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940adc00 0x7ff69415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff692000640 1 -- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940adc00 msgr2=0x7ff69415ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff692000640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940adc00 0x7ff69415ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff692000640 1 -- 172.21.15.150:0/2103827113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6940ae550 con 0x7ff6940af190 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff692000640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6940af190 0x7ff694158c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00d480 tx=0x7ff67c00d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff68b400640 1 -- 172.21.15.150:0/2103827113 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c005480 con 0x7ff6940af190 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff68b400640 1 -- 172.21.15.150:0/2103827113 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff67c017070 con 0x7ff6940af190 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff68b400640 1 -- 172.21.15.150:0/2103827113 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c003810 con 0x7ff6940af190 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6940ae830 con 0x7ff6940af190 2024-04-15T15:24:51.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.241+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69413ec20 con 0x7ff6940af190 2024-04-15T15:24:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.243+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff694068c80 con 0x7ff6940af190 2024-04-15T15:24:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.244+0000 7ff68b400640 1 -- 172.21.15.150:0/2103827113 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff67c0039b0 con 0x7ff6940af190 2024-04-15T15:24:51.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.244+0000 7ff68b400640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff67006c920 0x7ff67006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.244+0000 7ff68b400640 1 -- 172.21.15.150:0/2103827113 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7ff67c08bae0 con 0x7ff6940af190 2024-04-15T15:24:51.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.245+0000 7ff692a00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff67006c920 0x7ff67006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.246+0000 7ff692a00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff67006c920 0x7ff67006ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6840099a0 tx=0x7ff684008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:51.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.249+0000 7ff68b400640 1 -- 172.21.15.150:0/2103827113 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff67c059e80 con 0x7ff6940af190 2024-04-15T15:24:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.457+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff6940ccaa0 con 0x7ff6940af190 2024-04-15T15:24:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.458+0000 7ff68b400640 1 -- 172.21.15.150:0/2103827113 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7ff67c059780 con 0x7ff6940af190 2024-04-15T15:24:51.459 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:24:51.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.460+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff67006c920 msgr2=0x7ff67006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.460+0000 7ff699e00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff67006c920 0x7ff67006ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6840099a0 tx=0x7ff684008040 comp rx=0 tx=0).stop 2024-04-15T15:24:51.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.460+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6940af190 msgr2=0x7ff694158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.460+0000 7ff699e00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6940af190 0x7ff694158c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00d480 tx=0x7ff67c00d940 comp rx=0 tx=0).stop 2024-04-15T15:24:51.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.461+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 shutdown_connections 2024-04-15T15:24:51.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.461+0000 7ff699e00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6940adc00 0x7ff69415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.461+0000 7ff699e00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6940af190 0x7ff694158c80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.461+0000 7ff699e00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff67006c920 0x7ff67006ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.461+0000 7ff699e00640 1 --2- 172.21.15.150:0/2103827113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6780051c0 0x7ff6940b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.461+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 >> 172.21.15.150:0/2103827113 conn(0x7ff6940b2460 msgr2=0x7ff6940b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:51.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.461+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 shutdown_connections 2024-04-15T15:24:51.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.461+0000 7ff699e00640 1 -- 172.21.15.150:0/2103827113 wait complete. 2024-04-15T15:24:51.483 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:24:51.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.690+0000 7f10fd600640 1 Processor -- start 2024-04-15T15:24:51.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.690+0000 7f10fd600640 1 -- start start 2024-04-15T15:24:51.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.691+0000 7f10fd600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f8145170 0x7f10f80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10fd600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f10f806f260 0x7f10f806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10fd600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10f80b4410 con 0x7f10f80ce070 2024-04-15T15:24:51.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10fd600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f10f8145550 con 0x7f10f806f260 2024-04-15T15:24:51.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10fd600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10f80b46f0 con 0x7f10f8145170 2024-04-15T15:24:51.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10f6000640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f10f80ce070 0x7f10f80b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56704/0 (socket says 172.21.15.150:56704) 2024-04-15T15:24:51.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10f6000640 1 -- 172.21.15.150:0/3443565793 learned_addr learned my addr 172.21.15.150:0/3443565793 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:51.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10f7400640 1 --2- 172.21.15.150:0/3443565793 >> v2:172.21.15.150:3300/0 conn(0x7f10f806f260 0x7f10f806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10f6a00640 1 --2- 172.21.15.150:0/3443565793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f8145170 0x7f10f80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.692+0000 7f10f5600640 1 -- 172.21.15.150:0/3443565793 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2448714526 0 0) 0x7f10f80b4410 con 0x7f10f80ce070 2024-04-15T15:24:51.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.693+0000 7f10f5600640 1 -- 172.21.15.150:0/3443565793 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10d8003520 con 0x7f10f80ce070 2024-04-15T15:24:51.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.693+0000 7f10f7400640 1 -- 172.21.15.150:0/3443565793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f8145170 msgr2=0x7f10f80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:51.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.693+0000 7f10f7400640 1 --2- 172.21.15.150:0/3443565793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f8145170 0x7f10f80cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.693+0000 7f10f7400640 1 -- 172.21.15.150:0/3443565793 >> v1:172.21.15.150:6789/0 conn(0x7f10f80ce070 legacy=0x7f10f80b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.693+0000 7f10f7400640 1 -- 172.21.15.150:0/3443565793 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f80b49d0 con 0x7f10f806f260 2024-04-15T15:24:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.694+0000 7f10f7400640 1 --2- 172.21.15.150:0/3443565793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f806f260 0x7f10f806f6f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097a0 tx=0x7f10f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc454b3681b14cb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.694+0000 7f10f5600640 1 -- 172.21.15.150:0/3443565793 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0035020 con 0x7f10f806f260 2024-04-15T15:24:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.694+0000 7f10f5600640 1 -- 172.21.15.150:0/3443565793 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f0031830 con 0x7f10f806f260 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.694+0000 7f10f5600640 1 -- 172.21.15.150:0/3443565793 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0031c00 con 0x7f10f806f260 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.695+0000 7f10fd600640 1 -- 172.21.15.150:0/3443565793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f806f260 msgr2=0x7f10f806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.695+0000 7f10fd600640 1 --2- 172.21.15.150:0/3443565793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f806f260 0x7f10f806f6f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097a0 tx=0x7f10f002b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.695+0000 7f10fd600640 1 -- 172.21.15.150:0/3443565793 shutdown_connections 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.695+0000 7f10fd600640 1 --2- 172.21.15.150:0/3443565793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f806f260 0x7f10f806f6f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.695+0000 7f10fd600640 1 --2- 172.21.15.150:0/3443565793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f8145170 0x7f10f80cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.695+0000 7f10fd600640 1 -- 172.21.15.150:0/3443565793 >> 172.21.15.150:0/3443565793 conn(0x7f10f80b2460 msgr2=0x7f10f80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.695+0000 7f10fd600640 1 -- 172.21.15.150:0/3443565793 shutdown_connections 2024-04-15T15:24:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.695+0000 7f10fd600640 1 -- 172.21.15.150:0/3443565793 wait complete. 2024-04-15T15:24:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.696+0000 7f10fd600640 1 Processor -- start 2024-04-15T15:24:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.697+0000 7f10fd600640 1 -- start start 2024-04-15T15:24:51.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.697+0000 7f10fd600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f806f260 0x7f10f8158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.697+0000 7f10fd600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10f80ce070 0x7f10f815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.697+0000 7f10fd600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f8145170 0x7f10f815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.697+0000 7f10fd600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10f80b5bc0 con 0x7f10f8145170 2024-04-15T15:24:51.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.697+0000 7f10fd600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10f80b4f90 con 0x7f10f80ce070 2024-04-15T15:24:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.697+0000 7f10fd600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10f80b4030 con 0x7f10f806f260 2024-04-15T15:24:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f8145170 0x7f10f815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f8145170 0x7f10f815b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33672/0 (socket says 172.21.15.150:33672) 2024-04-15T15:24:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f7400640 1 -- 172.21.15.150:0/1844151329 learned_addr learned my addr 172.21.15.150:0/1844151329 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f6a00640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f806f260 0x7f10f8158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f7400640 1 -- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f806f260 msgr2=0x7f10f8158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f7400640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f806f260 0x7f10f8158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f7400640 1 -- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10f80ce070 msgr2=0x7f10f815ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f7400640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10f80ce070 0x7f10f815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.698+0000 7f10f7400640 1 -- 172.21.15.150:0/1844151329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f815b890 con 0x7f10f8145170 2024-04-15T15:24:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.699+0000 7f10f7400640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f8145170 0x7f10f815b350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f10f002bee0 tx=0x7f10f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.699+0000 7f10eea00640 1 -- 172.21.15.150:0/1844151329 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0035040 con 0x7f10f8145170 2024-04-15T15:24:51.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.699+0000 7f10eea00640 1 -- 172.21.15.150:0/1844151329 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f003f070 con 0x7f10f8145170 2024-04-15T15:24:51.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.699+0000 7f10eea00640 1 -- 172.21.15.150:0/1844151329 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f003ba80 con 0x7f10f8145170 2024-04-15T15:24:51.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.699+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f8159d70 con 0x7f10f8145170 2024-04-15T15:24:51.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.699+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f815a330 con 0x7f10f8145170 2024-04-15T15:24:51.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.700+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f8068c80 con 0x7f10f8145170 2024-04-15T15:24:51.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.701+0000 7f10eea00640 1 -- 172.21.15.150:0/1844151329 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10f0038020 con 0x7f10f8145170 2024-04-15T15:24:51.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.702+0000 7f10eea00640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f10d406c8f0 0x7f10d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:51.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.702+0000 7f10eea00640 1 -- 172.21.15.150:0/1844151329 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f10f00adba0 con 0x7f10f8145170 2024-04-15T15:24:51.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.703+0000 7f10f6a00640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f10d406c8f0 0x7f10d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.704+0000 7f10f6a00640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f10d406c8f0 0x7f10d406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f10e8006fd0 tx=0x7f10e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:51.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.706+0000 7f10eea00640 1 -- 172.21.15.150:0/1844151329 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10f0031800 con 0x7f10f8145170 2024-04-15T15:24:51.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.906+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10f8158d70 con 0x7f10f8145170 2024-04-15T15:24:51.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.907+0000 7f10eea00640 1 -- 172.21.15.150:0/1844151329 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f10f007be80 con 0x7f10f8145170 2024-04-15T15:24:51.909 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:24:51.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f10d406c8f0 msgr2=0x7f10d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f10d406c8f0 0x7f10d406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f10e8006fd0 tx=0x7f10e8008040 comp rx=0 tx=0).stop 2024-04-15T15:24:51.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f8145170 msgr2=0x7f10f815b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:51.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f8145170 0x7f10f815b350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f10f002bee0 tx=0x7f10f00046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 shutdown_connections 2024-04-15T15:24:51.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f8145170 0x7f10f815b350 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10f80ce070 0x7f10f815ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f10d406c8f0 0x7f10d406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 --2- 172.21.15.150:0/1844151329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10f806f260 0x7f10f8158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:51.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.910+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 >> 172.21.15.150:0/1844151329 conn(0x7f10f80b2460 msgr2=0x7f10f8143870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:51.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.911+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 shutdown_connections 2024-04-15T15:24:51.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:51.911+0000 7f10fd600640 1 -- 172.21.15.150:0/1844151329 wait complete. 2024-04-15T15:24:51.930 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:24:52.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.140+0000 7fde54e00640 1 Processor -- start 2024-04-15T15:24:52.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.140+0000 7fde54e00640 1 -- start start 2024-04-15T15:24:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.142+0000 7fde54e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde501481f0 0x7fde5015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.142+0000 7fde54e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde500affc0 0x7fde500721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.142+0000 7fde54e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde501485d0 con 0x7fde500b5560 2024-04-15T15:24:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.142+0000 7fde54e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fde5015ffc0 con 0x7fde501481f0 2024-04-15T15:24:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.142+0000 7fde54e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde500aef20 con 0x7fde500affc0 2024-04-15T15:24:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.142+0000 7fde4e800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fde500b5560 0x7fde500b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56714/0 (socket says 172.21.15.150:56714) 2024-04-15T15:24:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.142+0000 7fde4e800640 1 -- 172.21.15.150:0/3583891169 learned_addr learned my addr 172.21.15.150:0/3583891169 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4de00640 1 --2- 172.21.15.150:0/3583891169 >> v2:172.21.15.187:3300/0 conn(0x7fde501481f0 0x7fde5015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde47e00640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde500affc0 0x7fde500721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2792940092 0 0) 0x7fde501485d0 con 0x7fde500b5560 2024-04-15T15:24:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde34003520 con 0x7fde500b5560 2024-04-15T15:24:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3022224515 0 0) 0x7fde34003520 con 0x7fde500b5560 2024-04-15T15:24:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde500affc0 msgr2=0x7fde500721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde500affc0 0x7fde500721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 >> v2:172.21.15.187:3300/0 conn(0x7fde501481f0 msgr2=0x7fde5015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 --2- 172.21.15.150:0/3583891169 >> v2:172.21.15.187:3300/0 conn(0x7fde501481f0 0x7fde5015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde500af200 con 0x7fde500b5560 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fde48001a10 con 0x7fde500b5560 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 >> v1:172.21.15.150:6789/0 conn(0x7fde500b5560 legacy=0x7fde500b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde34004800 0x7fde34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde500affc0 0x7fde340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde500aef20 con 0x7fde340039b0 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde3400b9c0 con 0x7fde34004800 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.143+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde3400bb40 con 0x7fde500affc0 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.144+0000 7fde4de00640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.144+0000 7fde4de00640 1 -- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde500affc0 msgr2=0x7fde340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.144+0000 7fde4de00640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde500affc0 0x7fde340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.144+0000 7fde4de00640 1 -- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde34004800 msgr2=0x7fde34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:52.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.144+0000 7fde4de00640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde34004800 0x7fde34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.144+0000 7fde4de00640 1 -- 172.21.15.150:0/3583891169 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde34003520 con 0x7fde340039b0 2024-04-15T15:24:52.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.145+0000 7fde4de00640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde40009ac0 tx=0x7fde34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffc27e6a751e1f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:52.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.145+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40039070 con 0x7fde340039b0 2024-04-15T15:24:52.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.145+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4002c710 con 0x7fde340039b0 2024-04-15T15:24:52.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.145+0000 7fde4d400640 1 -- 172.21.15.150:0/3583891169 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40009690 con 0x7fde340039b0 2024-04-15T15:24:52.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.146+0000 7fde54e00640 1 -- 172.21.15.150:0/3583891169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde340039b0 msgr2=0x7fde34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.146+0000 7fde54e00640 1 --2- 172.21.15.150:0/3583891169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde40009ac0 tx=0x7fde34005830 comp rx=0 tx=0).stop 2024-04-15T15:24:52.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.146+0000 7fde4e800640 1 -- 172.21.15.150:0/3583891169 reap_dead start 2024-04-15T15:24:52.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.146+0000 7fde54e00640 1 -- 172.21.15.150:0/3583891169 shutdown_connections 2024-04-15T15:24:52.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.146+0000 7fde54e00640 1 -- 172.21.15.150:0/3583891169 >> 172.21.15.150:0/3583891169 conn(0x7fde500b27b0 msgr2=0x7fde500b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:52.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.146+0000 7fde54e00640 1 -- 172.21.15.150:0/3583891169 shutdown_connections 2024-04-15T15:24:52.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.146+0000 7fde54e00640 1 -- 172.21.15.150:0/3583891169 wait complete. 2024-04-15T15:24:52.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.148+0000 7fde54e00640 1 Processor -- start 2024-04-15T15:24:52.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.148+0000 7fde54e00640 1 -- start start 2024-04-15T15:24:52.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde54e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde340039b0 0x7fde501f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde54e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde5013b9f0 0x7fde5015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde54e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde5013be50 0x7fde5015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde54e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde500afaa0 con 0x7fde5013b9f0 2024-04-15T15:24:52.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde54e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde500af7c0 con 0x7fde5013be50 2024-04-15T15:24:52.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde54e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde500b3fe0 con 0x7fde340039b0 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde47e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde5013b9f0 0x7fde5015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde47e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde5013b9f0 0x7fde5015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33698/0 (socket says 172.21.15.150:33698) 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.149+0000 7fde47e00640 1 -- 172.21.15.150:0/327535789 learned_addr learned my addr 172.21.15.150:0/327535789 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde47e00640 1 -- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde340039b0 msgr2=0x7fde501f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde47e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde340039b0 0x7fde501f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde47e00640 1 -- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde5013be50 msgr2=0x7fde5015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde47e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde5013be50 0x7fde5015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde47e00640 1 -- 172.21.15.150:0/327535789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5013e290 con 0x7fde5013b9f0 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde47e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde5013b9f0 0x7fde5015de60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde3800e7d0 tx=0x7fde3800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde46000640 1 -- 172.21.15.150:0/327535789 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38017070 con 0x7fde5013b9f0 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde46000640 1 -- 172.21.15.150:0/327535789 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3800c940 con 0x7fde5013b9f0 2024-04-15T15:24:52.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde46000640 1 -- 172.21.15.150:0/327535789 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3800cd10 con 0x7fde5013b9f0 2024-04-15T15:24:52.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.150+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5013e570 con 0x7fde5013b9f0 2024-04-15T15:24:52.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.151+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde5013eb30 con 0x7fde5013b9f0 2024-04-15T15:24:52.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.152+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde50068c80 con 0x7fde5013b9f0 2024-04-15T15:24:52.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.152+0000 7fde46000640 1 -- 172.21.15.150:0/327535789 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde3801a070 con 0x7fde5013b9f0 2024-04-15T15:24:52.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.153+0000 7fde46000640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fde2c06c920 0x7fde2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.153+0000 7fde46000640 1 -- 172.21.15.150:0/327535789 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fde3808be30 con 0x7fde5013b9f0 2024-04-15T15:24:52.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.154+0000 7fde4de00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fde2c06c920 0x7fde2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.155+0000 7fde4de00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fde2c06c920 0x7fde2c06ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fde40036000 tx=0x7fde4002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.158+0000 7fde46000640 1 -- 172.21.15.150:0/327535789 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde3805a1b0 con 0x7fde5013b9f0 2024-04-15T15:24:52.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.390+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde5013ed30 con 0x7fde5013b9f0 2024-04-15T15:24:52.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.391+0000 7fde46000640 1 -- 172.21.15.150:0/327535789 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7fde38059ab0 con 0x7fde5013b9f0 2024-04-15T15:24:52.393 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:52.393 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:23:54.683053+0000","modified":"2024-04-15T15:23:57.779875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4250,"qdb_cluster":[4250]},"id":1}]} 2024-04-15T15:24:52.394 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fde2c06c920 msgr2=0x7fde2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fde2c06c920 0x7fde2c06ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fde40036000 tx=0x7fde4002e000 comp rx=0 tx=0).stop 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde5013b9f0 msgr2=0x7fde5015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde5013b9f0 0x7fde5015de60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde3800e7d0 tx=0x7fde3800ec90 comp rx=0 tx=0).stop 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 shutdown_connections 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde5013be50 0x7fde5015e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde5013b9f0 0x7fde5015de60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fde2c06c920 0x7fde2c06ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 --2- 172.21.15.150:0/327535789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde340039b0 0x7fde501f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 >> 172.21.15.150:0/327535789 conn(0x7fde500b27b0 msgr2=0x7fde50161940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:52.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 shutdown_connections 2024-04-15T15:24:52.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.394+0000 7fde54e00640 1 -- 172.21.15.150:0/327535789 wait complete. 2024-04-15T15:24:52.413 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:52.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.622+0000 7fbf97e00640 1 Processor -- start 2024-04-15T15:24:52.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.622+0000 7fbf97e00640 1 -- start start 2024-04-15T15:24:52.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf97e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf901461f0 0x7fbf900ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf97e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbf900cedf0 0x7fbf900b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf97e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900b3e10 0x7fbf900cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf97e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fbf900a9640 con 0x7fbf901461f0 2024-04-15T15:24:52.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf97e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbf900cf250 con 0x7fbf900cedf0 2024-04-15T15:24:52.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf97e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf900b4be0 con 0x7fbf900b3e10 2024-04-15T15:24:52.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf95400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf901461f0 0x7fbf900ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf95400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf901461f0 0x7fbf900ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33714/0 (socket says 172.21.15.150:33714) 2024-04-15T15:24:52.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf95400640 1 -- 172.21.15.150:0/1450828780 learned_addr learned my addr 172.21.15.150:0/1450828780 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:52.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf95e00640 1 --2- 172.21.15.150:0/1450828780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900b3e10 0x7fbf900cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.624+0000 7fbf94a00640 1 --2- 172.21.15.150:0/1450828780 >> v2:172.21.15.187:3300/0 conn(0x7fbf900cedf0 0x7fbf900b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.625+0000 7fbf95400640 1 -- 172.21.15.150:0/1450828780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900b3e10 msgr2=0x7fbf900cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:52.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.625+0000 7fbf95400640 1 --2- 172.21.15.150:0/1450828780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900b3e10 0x7fbf900cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.625+0000 7fbf95400640 1 -- 172.21.15.150:0/1450828780 >> v2:172.21.15.187:3300/0 conn(0x7fbf900cedf0 msgr2=0x7fbf900b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:52.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.625+0000 7fbf95400640 1 --2- 172.21.15.150:0/1450828780 >> v2:172.21.15.187:3300/0 conn(0x7fbf900cedf0 0x7fbf900b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.625+0000 7fbf95400640 1 -- 172.21.15.150:0/1450828780 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf900b4620 con 0x7fbf901461f0 2024-04-15T15:24:52.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.625+0000 7fbf94a00640 1 --2- 172.21.15.150:0/1450828780 >> v2:172.21.15.187:3300/0 conn(0x7fbf900cedf0 0x7fbf900b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.626+0000 7fbf95400640 1 --2- 172.21.15.150:0/1450828780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf900ce8b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbf90073560 tx=0x7fbf8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f263a9bb90adcd77 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.626+0000 7fbf8fe00640 1 -- 172.21.15.150:0/1450828780 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84034070 con 0x7fbf901461f0 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.626+0000 7fbf8fe00640 1 -- 172.21.15.150:0/1450828780 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf84030860 con 0x7fbf901461f0 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.626+0000 7fbf8fe00640 1 -- 172.21.15.150:0/1450828780 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84030c30 con 0x7fbf901461f0 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 -- 172.21.15.150:0/1450828780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 msgr2=0x7fbf900ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 --2- 172.21.15.150:0/1450828780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf900ce8b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbf90073560 tx=0x7fbf8402b9a0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 -- 172.21.15.150:0/1450828780 shutdown_connections 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 --2- 172.21.15.150:0/1450828780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900b3e10 0x7fbf900cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 --2- 172.21.15.150:0/1450828780 >> v2:172.21.15.187:3300/0 conn(0x7fbf900cedf0 0x7fbf900b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 --2- 172.21.15.150:0/1450828780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf900ce8b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 -- 172.21.15.150:0/1450828780 >> 172.21.15.150:0/1450828780 conn(0x7fbf900b27b0 msgr2=0x7fbf900b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:52.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 -- 172.21.15.150:0/1450828780 shutdown_connections 2024-04-15T15:24:52.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.627+0000 7fbf97e00640 1 -- 172.21.15.150:0/1450828780 wait complete. 2024-04-15T15:24:52.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.629+0000 7fbf97e00640 1 Processor -- start 2024-04-15T15:24:52.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.629+0000 7fbf97e00640 1 -- start start 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf97e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf900b3e10 0x7fbf9013ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf97e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900cedf0 0x7fbf9013fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf97e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf901f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf97e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9015d290 con 0x7fbf901461f0 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf97e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9015cfb0 con 0x7fbf900b3e10 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf97e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf901465d0 con 0x7fbf900cedf0 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf95e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf901f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf95e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf901f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33716/0 (socket says 172.21.15.150:33716) 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf95e00640 1 -- 172.21.15.150:0/966514613 learned_addr learned my addr 172.21.15.150:0/966514613 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf94a00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900cedf0 0x7fbf9013fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf95400640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf900b3e10 0x7fbf9013ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf95e00640 1 -- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900cedf0 msgr2=0x7fbf9013fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf95e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900cedf0 0x7fbf9013fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.630+0000 7fbf95e00640 1 -- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf900b3e10 msgr2=0x7fbf9013ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf95e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf900b3e10 0x7fbf9013ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf95e00640 1 -- 172.21.15.150:0/966514613 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9006ec50 con 0x7fbf901461f0 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf94a00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900cedf0 0x7fbf9013fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf95400640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf900b3e10 0x7fbf9013ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf95e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf901f7cf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbf88007e60 tx=0x7fbf8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf8d600640 1 -- 172.21.15.150:0/966514613 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88014620 con 0x7fbf901461f0 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf8d600640 1 -- 172.21.15.150:0/966514613 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf88019070 con 0x7fbf901461f0 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf900ce030 con 0x7fbf901461f0 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf8d600640 1 -- 172.21.15.150:0/966514613 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88014a20 con 0x7fbf901461f0 2024-04-15T15:24:52.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.631+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9006f1f0 con 0x7fbf901461f0 2024-04-15T15:24:52.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.632+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf900704e0 con 0x7fbf901461f0 2024-04-15T15:24:52.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.633+0000 7fbf8d600640 1 -- 172.21.15.150:0/966514613 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf88014bc0 con 0x7fbf901461f0 2024-04-15T15:24:52.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.634+0000 7fbf8d600640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf7006c920 0x7fbf7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:52.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.634+0000 7fbf8d600640 1 -- 172.21.15.150:0/966514613 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fbf8808c460 con 0x7fbf901461f0 2024-04-15T15:24:52.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.635+0000 7fbf95400640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf7006c920 0x7fbf7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:52.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.636+0000 7fbf95400640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf7006c920 0x7fbf7006ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf84036000 tx=0x7fbf8403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:52.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.639+0000 7fbf8d600640 1 -- 172.21.15.150:0/966514613 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf900704e0 con 0x7fbf901461f0 2024-04-15T15:24:52.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.839+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf90144b50 con 0x7fbf901461f0 2024-04-15T15:24:52.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.842+0000 7fbf8d600640 1 -- 172.21.15.150:0/966514613 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16997 (secure 0 0 0) 0x7fbf8805a7d0 con 0x7fbf901461f0 2024-04-15T15:24:52.844 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:52.844 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:36.500392+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.844+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf7006c920 msgr2=0x7fbf7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.844+0000 7fbf97e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf7006c920 0x7fbf7006ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf84036000 tx=0x7fbf8403b040 comp rx=0 tx=0).stop 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.844+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 msgr2=0x7fbf901f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.844+0000 7fbf97e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf901f7cf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbf88007e60 tx=0x7fbf8800e4b0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.845+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 shutdown_connections 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.845+0000 7fbf97e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf901461f0 0x7fbf901f7cf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.845+0000 7fbf97e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbf7006c920 0x7fbf7006ede0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.845+0000 7fbf97e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf900cedf0 0x7fbf9013fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.845+0000 7fbf97e00640 1 --2- 172.21.15.150:0/966514613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf900b3e10 0x7fbf9013ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:52.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.845+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 >> 172.21.15.150:0/966514613 conn(0x7fbf900b27b0 msgr2=0x7fbf90141070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:52.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.845+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 shutdown_connections 2024-04-15T15:24:52.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:52.845+0000 7fbf97e00640 1 -- 172.21.15.150:0/966514613 wait complete. 2024-04-15T15:24:52.867 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:24:52.867 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:24:53.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.066+0000 7fa284c00640 1 Processor -- start 2024-04-15T15:24:53.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.066+0000 7fa284c00640 1 -- start start 2024-04-15T15:24:53.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.068+0000 7fa284c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa2800b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.068+0000 7fa284c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa2800b5020 0x7fa28015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.068+0000 7fa284c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa28006f940 con 0x7fa28015b6d0 2024-04-15T15:24:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.068+0000 7fa284c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa2800b5480 con 0x7fa2800b5020 2024-04-15T15:24:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.068+0000 7fa284c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa28006e8a0 con 0x7fa280070210 2024-04-15T15:24:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.068+0000 7fa27d400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa2800b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.068+0000 7fa27e800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa2800b5020 0x7fa28015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.069+0000 7fa27d400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa2800b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:55622/0 (socket says 172.21.15.150:55622) 2024-04-15T15:24:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.069+0000 7fa27d400640 1 -- 172.21.15.150:0/1893905599 learned_addr learned my addr 172.21.15.150:0/1893905599 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.069+0000 7fa27d400640 1 -- 172.21.15.150:0/1893905599 >> v2:172.21.15.187:3300/0 conn(0x7fa2800b5020 msgr2=0x7fa28015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.069+0000 7fa27d400640 1 --2- 172.21.15.150:0/1893905599 >> v2:172.21.15.187:3300/0 conn(0x7fa2800b5020 0x7fa28015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.069+0000 7fa27d400640 1 -- 172.21.15.150:0/1893905599 >> v1:172.21.15.187:6789/0 conn(0x7fa28015b6d0 legacy=0x7fa2800cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.069+0000 7fa27d400640 1 -- 172.21.15.150:0/1893905599 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28006eb80 con 0x7fa280070210 2024-04-15T15:24:53.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.071+0000 7fa27d400640 1 --2- 172.21.15.150:0/1893905599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa2800b4ae0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009870 tx=0x7fa26c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fa1926d32aa5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:53.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.071+0000 7fa27ca00640 1 -- 172.21.15.150:0/1893905599 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c035020 con 0x7fa280070210 2024-04-15T15:24:53.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.071+0000 7fa27ca00640 1 -- 172.21.15.150:0/1893905599 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26c031640 con 0x7fa280070210 2024-04-15T15:24:53.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.071+0000 7fa27ca00640 1 -- 172.21.15.150:0/1893905599 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c02d040 con 0x7fa280070210 2024-04-15T15:24:53.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.072+0000 7fa284c00640 1 -- 172.21.15.150:0/1893905599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 msgr2=0x7fa2800b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.072+0000 7fa284c00640 1 --2- 172.21.15.150:0/1893905599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa2800b4ae0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009870 tx=0x7fa26c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.072+0000 7fa284c00640 1 -- 172.21.15.150:0/1893905599 shutdown_connections 2024-04-15T15:24:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.072+0000 7fa284c00640 1 --2- 172.21.15.150:0/1893905599 >> v2:172.21.15.187:3300/0 conn(0x7fa2800b5020 0x7fa28015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.072+0000 7fa284c00640 1 --2- 172.21.15.150:0/1893905599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa2800b4ae0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.072+0000 7fa284c00640 1 -- 172.21.15.150:0/1893905599 >> 172.21.15.150:0/1893905599 conn(0x7fa2800b27b0 msgr2=0x7fa2800b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.072+0000 7fa284c00640 1 -- 172.21.15.150:0/1893905599 shutdown_connections 2024-04-15T15:24:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.072+0000 7fa284c00640 1 -- 172.21.15.150:0/1893905599 wait complete. 2024-04-15T15:24:53.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.074+0000 7fa284c00640 1 Processor -- start 2024-04-15T15:24:53.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.074+0000 7fa284c00640 1 -- start start 2024-04-15T15:24:53.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa284c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa280143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa284c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2800b5020 0x7fa28014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa284c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa28015b6d0 0x7fa2801f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa284c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa28006f420 con 0x7fa2800b5020 2024-04-15T15:24:53.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa284c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa28006f140 con 0x7fa28015b6d0 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa284c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2800ce420 con 0x7fa280070210 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa27d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2800b5020 0x7fa28014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa27d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2800b5020 0x7fa28014ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33718/0 (socket says 172.21.15.150:33718) 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa27d400640 1 -- 172.21.15.150:0/2158849353 learned_addr learned my addr 172.21.15.150:0/2158849353 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa27de00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa280143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.075+0000 7fa27e800640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa28015b6d0 0x7fa2801f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa27d400640 1 -- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 msgr2=0x7fa280143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa27d400640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa280143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa27d400640 1 -- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa28015b6d0 msgr2=0x7fa2801f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa27d400640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa28015b6d0 0x7fa2801f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa27d400640 1 -- 172.21.15.150:0/2158849353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28014dcb0 con 0x7fa2800b5020 2024-04-15T15:24:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa27d400640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2800b5020 0x7fa28014ed50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa26c000c00 tx=0x7fa26c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:53.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa276000640 1 -- 172.21.15.150:0/2158849353 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c040070 con 0x7fa2800b5020 2024-04-15T15:24:53.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa276000640 1 -- 172.21.15.150:0/2158849353 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26c0399f0 con 0x7fa2800b5020 2024-04-15T15:24:53.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa276000640 1 -- 172.21.15.150:0/2158849353 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c035040 con 0x7fa2800b5020 2024-04-15T15:24:53.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.076+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28014df90 con 0x7fa2800b5020 2024-04-15T15:24:53.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.077+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28014e550 con 0x7fa2800b5020 2024-04-15T15:24:53.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.078+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa280068c80 con 0x7fa2800b5020 2024-04-15T15:24:53.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.078+0000 7fa276000640 1 -- 172.21.15.150:0/2158849353 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa26c04b060 con 0x7fa2800b5020 2024-04-15T15:24:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.079+0000 7fa276000640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa26006c920 0x7fa26006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.079+0000 7fa276000640 1 -- 172.21.15.150:0/2158849353 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fa26c0ad400 con 0x7fa2800b5020 2024-04-15T15:24:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.080+0000 7fa27de00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa26006c920 0x7fa26006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.081+0000 7fa27de00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa26006c920 0x7fa26006ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa270007fd0 tx=0x7fa270009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.084+0000 7fa276000640 1 -- 172.21.15.150:0/2158849353 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa26c07b810 con 0x7fa2800b5020 2024-04-15T15:24:53.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.328+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2800d0560 con 0x7fa2800b5020 2024-04-15T15:24:53.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.329+0000 7fa276000640 1 -- 172.21.15.150:0/2158849353 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7fa26c07b110 con 0x7fa2800b5020 2024-04-15T15:24:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:53.331 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:23:54.683053+0000","modified":"2024-04-15T15:23:57.779875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4250,"qdb_cluster":[4250]},"id":1}]} 2024-04-15T15:24:53.331 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-04-15T15:24:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa26006c920 msgr2=0x7fa26006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa26006c920 0x7fa26006ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa270007fd0 tx=0x7fa270009040 comp rx=0 tx=0).stop 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2800b5020 msgr2=0x7fa28014ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2800b5020 0x7fa28014ed50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa26c000c00 tx=0x7fa26c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 shutdown_connections 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa28015b6d0 0x7fa2801f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2800b5020 0x7fa28014ed50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa26006c920 0x7fa26006ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 --2- 172.21.15.150:0/2158849353 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa280070210 0x7fa280143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 >> 172.21.15.150:0/2158849353 conn(0x7fa2800b27b0 msgr2=0x7fa280153220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 shutdown_connections 2024-04-15T15:24:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.332+0000 7fa284c00640 1 -- 172.21.15.150:0/2158849353 wait complete. 2024-04-15T15:24:53.352 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:24:53.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.548+0000 7fe23a600640 1 Processor -- start 2024-04-15T15:24:53.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.548+0000 7fe23a600640 1 -- start start 2024-04-15T15:24:53.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.549+0000 7fe23a600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe2340cee00 0x7fe23406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.549+0000 7fe23a600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe23415d2f0 0x7fe2340723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.549+0000 7fe23a600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe23415dbb0 con 0x7fe234071f50 2024-04-15T15:24:53.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.549+0000 7fe23a600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fe23406f4f0 con 0x7fe2340cee00 2024-04-15T15:24:53.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.549+0000 7fe23a600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe2340728f0 con 0x7fe23415d2f0 2024-04-15T15:24:53.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.550+0000 7fe239200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe2340cee00 0x7fe23406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.550+0000 7fe233e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe234071f50 0x7fe23415cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56716/0 (socket says 172.21.15.150:56716) 2024-04-15T15:24:53.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.550+0000 7fe233e00640 1 -- 172.21.15.150:0/1003846513 learned_addr learned my addr 172.21.15.150:0/1003846513 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:53.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.550+0000 7fe239c00640 1 --2- 172.21.15.150:0/1003846513 >> v2:172.21.15.187:3300/0 conn(0x7fe23415d2f0 0x7fe2340723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.551+0000 7fe239200640 1 -- 172.21.15.150:0/1003846513 >> v2:172.21.15.187:3300/0 conn(0x7fe23415d2f0 msgr2=0x7fe2340723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:53.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.551+0000 7fe239200640 1 --2- 172.21.15.150:0/1003846513 >> v2:172.21.15.187:3300/0 conn(0x7fe23415d2f0 0x7fe2340723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.551+0000 7fe239200640 1 -- 172.21.15.150:0/1003846513 >> v1:172.21.15.150:6789/0 conn(0x7fe234071f50 legacy=0x7fe23415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.551+0000 7fe233400640 1 -- 172.21.15.150:0/1003846513 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173701956 0 0) 0x7fe23415dbb0 con 0x7fe234071f50 2024-04-15T15:24:53.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.551+0000 7fe239200640 1 -- 172.21.15.150:0/1003846513 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23415d8d0 con 0x7fe2340cee00 2024-04-15T15:24:53.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.552+0000 7fe239200640 1 --2- 172.21.15.150:0/1003846513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe23406e8c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe23415a7b0 tx=0x7fe22802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbd8839a1829111 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:53.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.552+0000 7fe233400640 1 -- 172.21.15.150:0/1003846513 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228035020 con 0x7fe2340cee00 2024-04-15T15:24:53.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.552+0000 7fe233400640 1 -- 172.21.15.150:0/1003846513 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe228031830 con 0x7fe2340cee00 2024-04-15T15:24:53.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.552+0000 7fe233400640 1 -- 172.21.15.150:0/1003846513 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228031c00 con 0x7fe2340cee00 2024-04-15T15:24:53.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.552+0000 7fe23a600640 1 -- 172.21.15.150:0/1003846513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 msgr2=0x7fe23406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.552+0000 7fe23a600640 1 --2- 172.21.15.150:0/1003846513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe23406e8c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe23415a7b0 tx=0x7fe22802b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.553+0000 7fe23a600640 1 -- 172.21.15.150:0/1003846513 shutdown_connections 2024-04-15T15:24:53.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.553+0000 7fe23a600640 1 --2- 172.21.15.150:0/1003846513 >> v2:172.21.15.187:3300/0 conn(0x7fe23415d2f0 0x7fe2340723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.553+0000 7fe23a600640 1 --2- 172.21.15.150:0/1003846513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe23406e8c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.553+0000 7fe23a600640 1 -- 172.21.15.150:0/1003846513 >> 172.21.15.150:0/1003846513 conn(0x7fe23406b930 msgr2=0x7fe23406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:53.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.553+0000 7fe23a600640 1 -- 172.21.15.150:0/1003846513 shutdown_connections 2024-04-15T15:24:53.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.553+0000 7fe23a600640 1 -- 172.21.15.150:0/1003846513 wait complete. 2024-04-15T15:24:53.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.554+0000 7fe23a600640 1 Processor -- start 2024-04-15T15:24:53.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.555+0000 7fe23a600640 1 -- start start 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.555+0000 7fe23a600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe234071f50 0x7fe234152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.555+0000 7fe23a600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe234154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.555+0000 7fe23a600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe23415d2f0 0x7fe23415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.555+0000 7fe23a600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe23406e3a0 con 0x7fe2340cee00 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.555+0000 7fe23a600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe23406e0c0 con 0x7fe234071f50 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.555+0000 7fe23a600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe23415dda0 con 0x7fe23415d2f0 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe234154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe234154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33734/0 (socket says 172.21.15.150:33734) 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 -- 172.21.15.150:0/764705359 learned_addr learned my addr 172.21.15.150:0/764705359 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:53.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe239200640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe234071f50 0x7fe234152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 -- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe23415d2f0 msgr2=0x7fe23415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe23415d2f0 0x7fe23415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 -- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe234071f50 msgr2=0x7fe234152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe234071f50 0x7fe234152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 -- 172.21.15.150:0/764705359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe234152f80 con 0x7fe2340cee00 2024-04-15T15:24:53.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.556+0000 7fe233e00640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe234154e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe22000c4a0 tx=0x7fe22000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:53.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.557+0000 7fe230c00640 1 -- 172.21.15.150:0/764705359 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220018070 con 0x7fe2340cee00 2024-04-15T15:24:53.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.557+0000 7fe230c00640 1 -- 172.21.15.150:0/764705359 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2200074e0 con 0x7fe2340cee00 2024-04-15T15:24:53.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.557+0000 7fe230c00640 1 -- 172.21.15.150:0/764705359 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220004a20 con 0x7fe2340cee00 2024-04-15T15:24:53.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.557+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe234153260 con 0x7fe2340cee00 2024-04-15T15:24:53.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.557+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe234153820 con 0x7fe2340cee00 2024-04-15T15:24:53.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.558+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe234068d90 con 0x7fe2340cee00 2024-04-15T15:24:53.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.559+0000 7fe230c00640 1 -- 172.21.15.150:0/764705359 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe22001b070 con 0x7fe2340cee00 2024-04-15T15:24:53.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.560+0000 7fe230c00640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe21406c8f0 0x7fe21406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:53.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.560+0000 7fe230c00640 1 -- 172.21.15.150:0/764705359 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fe22008afe0 con 0x7fe2340cee00 2024-04-15T15:24:53.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.560+0000 7fe239200640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe21406c8f0 0x7fe21406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:53.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.562+0000 7fe239200640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe21406c8f0 0x7fe21406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe228036000 tx=0x7fe228008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:53.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.565+0000 7fe230c00640 1 -- 172.21.15.150:0/764705359 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe234068d90 con 0x7fe2340cee00 2024-04-15T15:24:53.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.805+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2340755c0 con 0x7fe2340cee00 2024-04-15T15:24:53.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.806+0000 7fe230c00640 1 -- 172.21.15.150:0/764705359 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7fe2200592e0 con 0x7fe2340cee00 2024-04-15T15:24:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:53.808 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2749074573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2749074573},{"type":"v1","addr":"172.21.15.150:6835","nonce":2749074573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3795358390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3795358390},{"type":"v1","addr":"172.21.15.187:6836","nonce":3795358390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1416405850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1416405850},{"type":"v1","addr":"172.21.15.150:6834","nonce":1416405850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:23:54.683053+0000","modified":"2024-04-15T15:23:57.779875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/3441543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3441543802},{"type":"v1","addr":"172.21.15.187:6837","nonce":3441543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4250,"qdb_cluster":[4250]},"id":1}]} 2024-04-15T15:24:53.808 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-04-15T15:24:53.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe21406c8f0 msgr2=0x7fe21406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe21406c8f0 0x7fe21406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe228036000 tx=0x7fe228008030 comp rx=0 tx=0).stop 2024-04-15T15:24:53.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 msgr2=0x7fe234154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:53.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe234154e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe22000c4a0 tx=0x7fe22000c960 comp rx=0 tx=0).stop 2024-04-15T15:24:53.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 shutdown_connections 2024-04-15T15:24:53.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe21406c8f0 0x7fe21406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe23415d2f0 0x7fe23415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2340cee00 0x7fe234154e60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 --2- 172.21.15.150:0/764705359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe234071f50 0x7fe234152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:53.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 >> 172.21.15.150:0/764705359 conn(0x7fe23406b930 msgr2=0x7fe2341583e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:53.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 shutdown_connections 2024-04-15T15:24:53.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:53.809+0000 7fe23a600640 1 -- 172.21.15.150:0/764705359 wait complete. 2024-04-15T15:24:53.829 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:24:54.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.030+0000 7f9c13200640 1 Processor -- start 2024-04-15T15:24:54.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.030+0000 7f9c13200640 1 -- start start 2024-04-15T15:24:54.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.031+0000 7f9c13200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0c0cee00 0x7f9c0c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.031+0000 7f9c13200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c15dbb0 con 0x7f9c0c15d2f0 2024-04-15T15:24:54.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.031+0000 7f9c13200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c15d8d0 con 0x7f9c0c071f50 2024-04-15T15:24:54.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.031+0000 7f9c13200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c06f4f0 con 0x7f9c0c0cee00 2024-04-15T15:24:54.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c12800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9c0c15d2f0 0x7f9c0c15dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56726/0 (socket says 172.21.15.150:56726) 2024-04-15T15:24:54.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c12800640 1 -- 172.21.15.150:0/2894270440 learned_addr learned my addr 172.21.15.150:0/2894270440 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c11e00640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0c0cee00 0x7f9c0c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084719781 0 0) 0x7f9c0c15dbb0 con 0x7f9c0c15d2f0 2024-04-15T15:24:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bf4003520 con 0x7f9c0c15d2f0 2024-04-15T15:24:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2727601060 0 0) 0x7f9bf4003520 con 0x7f9c0c15d2f0 2024-04-15T15:24:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0c0cee00 msgr2=0x7f9c0c06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0c0cee00 0x7f9c0c06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 >> v1:172.21.15.187:6789/0 conn(0x7f9c0c071f50 legacy=0x7f9c0c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c06dde0 con 0x7f9c0c15d2f0 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f9c080019f0 con 0x7f9c0c15d2f0 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 >> v1:172.21.15.150:6789/0 conn(0x7f9c0c15d2f0 legacy=0x7f9c0c15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.032+0000 7f9c10a00640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0c0cee00 0x7f9bf40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c10a00640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4004010 0x7f9bf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c10a00640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bf4004d40 0x7f9bf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf400bb50 con 0x7f9bf4004010 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf400bd00 con 0x7f9bf4004d40 2024-04-15T15:24:54.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf400bf00 con 0x7f9c0c0cee00 2024-04-15T15:24:54.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c11400640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4004010 0x7f9bf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c11e00640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0c0cee00 0x7f9bf40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c11400640 1 -- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0c0cee00 msgr2=0x7f9bf40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c11400640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c0c0cee00 0x7f9bf40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c11400640 1 -- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bf4004d40 msgr2=0x7f9bf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c11400640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bf4004d40 0x7f9bf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.033+0000 7f9c11400640 1 -- 172.21.15.150:0/2894270440 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf4003520 con 0x7f9bf4004010 2024-04-15T15:24:54.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.034+0000 7f9c11400640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4004010 0x7f9bf4004800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc001750 tx=0x7f9bf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8a7546e40ba8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc001370 con 0x7f9bf4004010 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bfc0052d0 con 0x7f9bf4004010 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c10a00640 1 -- 172.21.15.150:0/2894270440 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc003830 con 0x7f9bf4004010 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c13200640 1 -- 172.21.15.150:0/2894270440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4004010 msgr2=0x7f9bf4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c13200640 1 --2- 172.21.15.150:0/2894270440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bf4004010 0x7f9bf4004800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc001750 tx=0x7f9bf4005830 comp rx=0 tx=0).stop 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c12800640 1 -- 172.21.15.150:0/2894270440 reap_dead start 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c13200640 1 -- 172.21.15.150:0/2894270440 shutdown_connections 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c13200640 1 -- 172.21.15.150:0/2894270440 >> 172.21.15.150:0/2894270440 conn(0x7f9c0c06b930 msgr2=0x7f9c0c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c13200640 1 -- 172.21.15.150:0/2894270440 shutdown_connections 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.035+0000 7f9c13200640 1 -- 172.21.15.150:0/2894270440 wait complete. 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.037+0000 7f9c13200640 1 Processor -- start 2024-04-15T15:24:54.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.037+0000 7f9c13200640 1 -- start start 2024-04-15T15:24:54.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c13200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf4004010 0x7f9c0c1f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c13200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c0c0b19b0 0x7f9c0c14ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c13200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0c150ad0 0x7f9c0c15f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c13200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c06e8c0 con 0x7f9c0c150ad0 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c13200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c06e3a0 con 0x7f9c0c0b19b0 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c13200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c06ea40 con 0x7f9bf4004010 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c12800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0c150ad0 0x7f9c0c15f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c12800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0c150ad0 0x7f9c0c15f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33748/0 (socket says 172.21.15.150:33748) 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.038+0000 7f9c12800640 1 -- 172.21.15.150:0/1674408821 learned_addr learned my addr 172.21.15.150:0/1674408821 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c11e00640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf4004010 0x7f9c0c1f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c11400640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c0c0b19b0 0x7f9c0c14ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c12800640 1 -- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf4004010 msgr2=0x7f9c0c1f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c12800640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf4004010 0x7f9c0c1f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c12800640 1 -- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c0c0b19b0 msgr2=0x7f9c0c14ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c12800640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c0c0b19b0 0x7f9c0c14ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c12800640 1 -- 172.21.15.150:0/1674408821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1f32d0 con 0x7f9c0c150ad0 2024-04-15T15:24:54.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c12800640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0c150ad0 0x7f9c0c15f3d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c840 tx=0x7f9c0800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c06000640 1 -- 172.21.15.150:0/1674408821 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800a640 con 0x7f9c0c150ad0 2024-04-15T15:24:54.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c06000640 1 -- 172.21.15.150:0/1674408821 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0800a7e0 con 0x7f9c0c150ad0 2024-04-15T15:24:54.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c06000640 1 -- 172.21.15.150:0/1674408821 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800abd0 con 0x7f9c0c150ad0 2024-04-15T15:24:54.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.039+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c159f30 con 0x7f9c0c150ad0 2024-04-15T15:24:54.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.040+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c15dbb0 con 0x7f9c0c150ad0 2024-04-15T15:24:54.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.041+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c159070 con 0x7f9c0c150ad0 2024-04-15T15:24:54.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.041+0000 7f9c06000640 1 -- 172.21.15.150:0/1674408821 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c08013450 con 0x7f9c0c150ad0 2024-04-15T15:24:54.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.042+0000 7f9c06000640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9bf006c920 0x7f9bf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.042+0000 7f9c06000640 1 -- 172.21.15.150:0/1674408821 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f9c0808b4c0 con 0x7f9c0c150ad0 2024-04-15T15:24:54.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.043+0000 7f9c11e00640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9bf006c920 0x7f9bf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.044+0000 7f9c11e00640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9bf006c920 0x7f9bf006ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c00006fd0 tx=0x7f9c00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.047+0000 7f9c06000640 1 -- 172.21.15.150:0/1674408821 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c0801a020 con 0x7f9c0c150ad0 2024-04-15T15:24:54.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.235+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c0c156160 con 0x7f9c0c150ad0 2024-04-15T15:24:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.237+0000 7f9c06000640 1 -- 172.21.15.150:0/1674408821 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16997 (secure 0 0 0) 0x7f9c08059150 con 0x7f9c0c150ad0 2024-04-15T15:24:54.239 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:54.239 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:36.500392+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:23:53.008505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-15T15:23:53.658759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.239+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9bf006c920 msgr2=0x7f9bf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.239+0000 7f9c13200640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9bf006c920 0x7f9bf006ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c00006fd0 tx=0x7f9c00008040 comp rx=0 tx=0).stop 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.239+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0c150ad0 msgr2=0x7f9c0c15f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.239+0000 7f9c13200640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0c150ad0 0x7f9c0c15f3d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c840 tx=0x7f9c0800cd00 comp rx=0 tx=0).stop 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.240+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 shutdown_connections 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.240+0000 7f9c13200640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0c150ad0 0x7f9c0c15f3d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.240+0000 7f9c13200640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c0c0b19b0 0x7f9c0c14ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.240+0000 7f9c13200640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9bf006c920 0x7f9bf006ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.240+0000 7f9c13200640 1 --2- 172.21.15.150:0/1674408821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bf4004010 0x7f9c0c1f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.240+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 >> 172.21.15.150:0/1674408821 conn(0x7f9c0c06b930 msgr2=0x7f9c0c1584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.240+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 shutdown_connections 2024-04-15T15:24:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.240+0000 7f9c13200640 1 -- 172.21.15.150:0/1674408821 wait complete. 2024-04-15T15:24:54.261 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:24:54.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.466+0000 7fc5c5c00640 1 Processor -- start 2024-04-15T15:24:54.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.466+0000 7fc5c5c00640 1 -- start start 2024-04-15T15:24:54.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.467+0000 7fc5c5c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 0x7fc5c00ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5c5c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc5c0072580 0x7fc5c006e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5c5c00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c015d200 con 0x7fc5c0072100 2024-04-15T15:24:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5c5c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c00a95b0 con 0x7fc5c0072580 2024-04-15T15:24:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5c5c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c015d4e0 con 0x7fc5c013f6a0 2024-04-15T15:24:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5b6000640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc5c0072100 0x7fc5c006e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56742/0 (socket says 172.21.15.150:56742) 2024-04-15T15:24:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5b6000640 1 -- 172.21.15.150:0/710074823 learned_addr learned my addr 172.21.15.150:0/710074823 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:54.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5bea00640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 0x7fc5c00ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3309719710 0 0) 0x7fc5c015d200 con 0x7fc5c0072100 2024-04-15T15:24:54.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5a4003520 con 0x7fc5c0072100 2024-04-15T15:24:54.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.468+0000 7fc5bf400640 1 --2- 172.21.15.150:0/710074823 >> v2:172.21.15.187:3300/0 conn(0x7fc5c0072580 0x7fc5c006e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1796225934 0 0) 0x7fc5a4003520 con 0x7fc5c0072100 2024-04-15T15:24:54.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 msgr2=0x7fc5c00ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 0x7fc5c00ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 >> v2:172.21.15.187:3300/0 conn(0x7fc5c0072580 msgr2=0x7fc5c006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 --2- 172.21.15.150:0/710074823 >> v2:172.21.15.187:3300/0 conn(0x7fc5c0072580 0x7fc5c006e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c015d7c0 con 0x7fc5c0072100 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fc5a8001670 con 0x7fc5c0072100 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 >> v1:172.21.15.150:6789/0 conn(0x7fc5c0072100 legacy=0x7fc5c006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 0x7fc5a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a4004d40 0x7fc5a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c015d4e0 con 0x7fc5a40039b0 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a400b9c0 con 0x7fc5a4004d40 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a400bb40 con 0x7fc5c013f6a0 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.469+0000 7fc5bea00640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5b6000640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 0x7fc5a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5bea00640 1 -- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 msgr2=0x7fc5a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5bea00640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 0x7fc5a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5bea00640 1 -- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a4004d40 msgr2=0x7fc5a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5bf400640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a4004d40 0x7fc5a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5bea00640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a4004d40 0x7fc5a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5bea00640 1 -- 172.21.15.150:0/710074823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a4003520 con 0x7fc5a40039b0 2024-04-15T15:24:54.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5b6000640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c013f6a0 0x7fc5a4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:54.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.470+0000 7fc5bf400640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a4004d40 0x7fc5a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:54.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.471+0000 7fc5bea00640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5a4003dd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009af0 tx=0x7fc5a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e2afd4a1845a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.471+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0039070 con 0x7fc5a40039b0 2024-04-15T15:24:54.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.471+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b002c6b0 con 0x7fc5a40039b0 2024-04-15T15:24:54.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.471+0000 7fc5be000640 1 -- 172.21.15.150:0/710074823 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0009690 con 0x7fc5a40039b0 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.472+0000 7fc5c5c00640 1 -- 172.21.15.150:0/710074823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 msgr2=0x7fc5a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.472+0000 7fc5c5c00640 1 --2- 172.21.15.150:0/710074823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5a4003dd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009af0 tx=0x7fc5a4005830 comp rx=0 tx=0).stop 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.472+0000 7fc5bf400640 1 -- 172.21.15.150:0/710074823 reap_dead start 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.472+0000 7fc5c5c00640 1 -- 172.21.15.150:0/710074823 shutdown_connections 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.472+0000 7fc5c5c00640 1 -- 172.21.15.150:0/710074823 >> 172.21.15.150:0/710074823 conn(0x7fc5c00b2460 msgr2=0x7fc5c00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.472+0000 7fc5c5c00640 1 -- 172.21.15.150:0/710074823 shutdown_connections 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.472+0000 7fc5c5c00640 1 -- 172.21.15.150:0/710074823 wait complete. 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.473+0000 7fc5c5c00640 1 Processor -- start 2024-04-15T15:24:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.474+0000 7fc5c5c00640 1 -- start start 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.474+0000 7fc5c5c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5c01f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.474+0000 7fc5c5c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c0156af0 0x7fc5c0141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.474+0000 7fc5c5c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c0141cc0 0x7fc5c00ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.474+0000 7fc5c5c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c013fef0 con 0x7fc5a40039b0 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.474+0000 7fc5c5c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00729f0 con 0x7fc5c0156af0 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.474+0000 7fc5c5c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c015dd80 con 0x7fc5c0141cc0 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5c01f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5c01f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33768/0 (socket says 172.21.15.150:33768) 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bea00640 1 -- 172.21.15.150:0/4177296850 learned_addr learned my addr 172.21.15.150:0/4177296850 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bf400640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c0141cc0 0x7fc5c00ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bea00640 1 -- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c0141cc0 msgr2=0x7fc5c00ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bea00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c0141cc0 0x7fc5c00ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5b6000640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c0156af0 0x7fc5c0141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bea00640 1 -- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c0156af0 msgr2=0x7fc5c0141780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bea00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c0156af0 0x7fc5c0141780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.475+0000 7fc5bea00640 1 -- 172.21.15.150:0/4177296850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c01f8c20 con 0x7fc5a40039b0 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.476+0000 7fc5bea00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5c01f86e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0037040 tx=0x7fc5b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.476+0000 7fc5b7400640 1 -- 172.21.15.150:0/4177296850 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0039050 con 0x7fc5a40039b0 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.476+0000 7fc5b7400640 1 -- 172.21.15.150:0/4177296850 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b0045070 con 0x7fc5a40039b0 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.476+0000 7fc5b7400640 1 -- 172.21.15.150:0/4177296850 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0003a50 con 0x7fc5a40039b0 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.476+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c01f4ac0 con 0x7fc5a40039b0 2024-04-15T15:24:54.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.476+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c01f5080 con 0x7fc5a40039b0 2024-04-15T15:24:54.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.477+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c015b0b0 con 0x7fc5a40039b0 2024-04-15T15:24:54.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.478+0000 7fc5b7400640 1 -- 172.21.15.150:0/4177296850 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5b0003bf0 con 0x7fc5a40039b0 2024-04-15T15:24:54.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.479+0000 7fc5b7400640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc59c06c920 0x7fc59c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.479+0000 7fc5b7400640 1 -- 172.21.15.150:0/4177296850 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7fc5b00ade40 con 0x7fc5a40039b0 2024-04-15T15:24:54.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.480+0000 7fc5b6000640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc59c06c920 0x7fc59c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.481+0000 7fc5b6000640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc59c06c920 0x7fc59c06ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00703b0 tx=0x7fc5a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.483+0000 7fc5b7400640 1 -- 172.21.15.150:0/4177296850 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5c015b0b0 con 0x7fc5a40039b0 2024-04-15T15:24:54.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.706+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc5c01420a0 con 0x7fc5a40039b0 2024-04-15T15:24:54.708 INFO:tasks.ceph.mon.a.smithi150.stderr:2024-04-15T15:24:54.707+0000 7f8c07200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:24:54.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.713+0000 7fc5b7400640 1 -- 172.21.15.150:0/4177296850 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fc5c01f8c20 con 0x7fc5a40039b0 2024-04-15T15:24:54.714 INFO:teuthology.orchestra.run.smithi150.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:24:54.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.715+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc59c06c920 msgr2=0x7fc59c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.715+0000 7fc5c5c00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc59c06c920 0x7fc59c06ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00703b0 tx=0x7fc5a8009040 comp rx=0 tx=0).stop 2024-04-15T15:24:54.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.715+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 msgr2=0x7fc5c01f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.715+0000 7fc5c5c00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5c01f86e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0037040 tx=0x7fc5b0004730 comp rx=0 tx=0).stop 2024-04-15T15:24:54.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.716+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 shutdown_connections 2024-04-15T15:24:54.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.716+0000 7fc5c5c00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc59c06c920 0x7fc59c06ede0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.716+0000 7fc5c5c00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c0141cc0 0x7fc5c00ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.716+0000 7fc5c5c00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c0156af0 0x7fc5c0141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.716+0000 7fc5c5c00640 1 --2- 172.21.15.150:0/4177296850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5a40039b0 0x7fc5c01f86e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.716+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 >> 172.21.15.150:0/4177296850 conn(0x7fc5c00b2460 msgr2=0x7fc5c015a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:54.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.716+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 shutdown_connections 2024-04-15T15:24:54.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.716+0000 7fc5c5c00640 1 -- 172.21.15.150:0/4177296850 wait complete. 2024-04-15T15:24:54.737 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-15T15:24:54.757 INFO:tasks.ceph.mds.b.smithi187.stderr: -8389> 2024-04-15T15:24:01.130+0000 7f606b000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:24:54.845 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-15T15:24:54.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.933+0000 7fc320a00640 1 Processor -- start 2024-04-15T15:24:54.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.933+0000 7fc320a00640 1 -- start start 2024-04-15T15:24:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.934+0000 7fc320a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc31c15c680 0x7fc31c162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.934+0000 7fc320a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc31c0b6c70 0x7fc31c15f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.934+0000 7fc320a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc31c0b70d0 con 0x7fc31c15fd90 2024-04-15T15:24:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.935+0000 7fc320a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc31c162dd0 con 0x7fc31c15c680 2024-04-15T15:24:54.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.935+0000 7fc320a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc31c1630b0 con 0x7fc31c0b6c70 2024-04-15T15:24:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.935+0000 7fc31b200640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc31c15fd90 0x7fc31c0d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56744/0 (socket says 172.21.15.150:56744) 2024-04-15T15:24:54.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.935+0000 7fc31b200640 1 -- 172.21.15.150:0/2072678272 learned_addr learned my addr 172.21.15.150:0/2072678272 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:54.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.935+0000 7fc31a800640 1 --2- 172.21.15.150:0/2072678272 >> v2:172.21.15.187:3300/0 conn(0x7fc31c15c680 0x7fc31c162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.935+0000 7fc319e00640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc31c0b6c70 0x7fc31c15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.935+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 878985566 0 0) 0x7fc31c0b70d0 con 0x7fc31c15fd90 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.935+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc300003520 con 0x7fc31c15fd90 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2849437991 0 0) 0x7fc300003520 con 0x7fc31c15fd90 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc31c0b6c70 msgr2=0x7fc31c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc31c0b6c70 0x7fc31c15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 >> v2:172.21.15.187:3300/0 conn(0x7fc31c15c680 msgr2=0x7fc31c162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 --2- 172.21.15.150:0/2072678272 >> v2:172.21.15.187:3300/0 conn(0x7fc31c15c680 0x7fc31c162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c163390 con 0x7fc31c15fd90 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fc3140019f0 con 0x7fc31c15fd90 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 >> v1:172.21.15.150:6789/0 conn(0x7fc31c15fd90 legacy=0x7fc31c0d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3000039b0 0x7fc300003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc300004800 0x7fc300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc31c0b6c70 0x7fc3000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c162dd0 con 0x7fc3000039b0 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1630b0 con 0x7fc300004800 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc30000b9c0 con 0x7fc31c0b6c70 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc31a800640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3000039b0 0x7fc300003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc31a800640 1 -- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc31c0b6c70 msgr2=0x7fc3000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.936+0000 7fc31a800640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc31c0b6c70 0x7fc3000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.937+0000 7fc31a800640 1 -- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc300004800 msgr2=0x7fc300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.937+0000 7fc31a800640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc300004800 0x7fc300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.937+0000 7fc31a800640 1 -- 172.21.15.150:0/2072678272 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc300003520 con 0x7fc3000039b0 2024-04-15T15:24:54.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.937+0000 7fc31a800640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3000039b0 0x7fc300003dd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ac0 tx=0x7fc300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf26b1220decfc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c039070 con 0x7fc3000039b0 2024-04-15T15:24:54.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30c02c710 con 0x7fc3000039b0 2024-04-15T15:24:54.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc319400640 1 -- 172.21.15.150:0/2072678272 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c009690 con 0x7fc3000039b0 2024-04-15T15:24:54.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc320a00640 1 -- 172.21.15.150:0/2072678272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3000039b0 msgr2=0x7fc300003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc320a00640 1 --2- 172.21.15.150:0/2072678272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3000039b0 0x7fc300003dd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ac0 tx=0x7fc300005830 comp rx=0 tx=0).stop 2024-04-15T15:24:54.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc31b200640 1 -- 172.21.15.150:0/2072678272 reap_dead start 2024-04-15T15:24:54.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc320a00640 1 -- 172.21.15.150:0/2072678272 shutdown_connections 2024-04-15T15:24:54.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc320a00640 1 -- 172.21.15.150:0/2072678272 >> 172.21.15.150:0/2072678272 conn(0x7fc31c073480 msgr2=0x7fc31c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:54.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.938+0000 7fc320a00640 1 -- 172.21.15.150:0/2072678272 shutdown_connections 2024-04-15T15:24:54.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.939+0000 7fc320a00640 1 -- 172.21.15.150:0/2072678272 wait complete. 2024-04-15T15:24:54.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.940+0000 7fc320a00640 1 Processor -- start 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.940+0000 7fc320a00640 1 -- start start 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.941+0000 7fc320a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3000039b0 0x7fc31c1f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.941+0000 7fc320a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31c0b2d60 0x7fc31c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.941+0000 7fc320a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc31c0c8b70 0x7fc31c160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.941+0000 7fc320a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c0d3260 con 0x7fc31c0b2d60 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.941+0000 7fc320a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c15ca60 con 0x7fc31c0c8b70 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.941+0000 7fc320a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc31c0d33e0 con 0x7fc3000039b0 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.941+0000 7fc319e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31c0b2d60 0x7fc31c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc319e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31c0b2d60 0x7fc31c15b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33788/0 (socket says 172.21.15.150:33788) 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc319e00640 1 -- 172.21.15.150:0/3910707811 learned_addr learned my addr 172.21.15.150:0/3910707811 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc31a800640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3000039b0 0x7fc31c1f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc31b200640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc31c0c8b70 0x7fc31c160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc319e00640 1 -- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3000039b0 msgr2=0x7fc31c1f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc319e00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3000039b0 0x7fc31c1f4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc319e00640 1 -- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc31c0c8b70 msgr2=0x7fc31c160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:54.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc319e00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc31c0c8b70 0x7fc31c160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:54.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc319e00640 1 -- 172.21.15.150:0/3910707811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c0cdc70 con 0x7fc31c0b2d60 2024-04-15T15:24:54.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc319e00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31c0b2d60 0x7fc31c15b670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc30800e7a0 tx=0x7fc30800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.942+0000 7fc312a00640 1 -- 172.21.15.150:0/3910707811 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308017070 con 0x7fc31c0b2d60 2024-04-15T15:24:54.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.943+0000 7fc312a00640 1 -- 172.21.15.150:0/3910707811 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30800c940 con 0x7fc31c0b2d60 2024-04-15T15:24:54.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.943+0000 7fc312a00640 1 -- 172.21.15.150:0/3910707811 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30800cd10 con 0x7fc31c0b2d60 2024-04-15T15:24:54.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.943+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c0b2a60 con 0x7fc31c0b2d60 2024-04-15T15:24:54.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.943+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c1f4a60 con 0x7fc31c0b2d60 2024-04-15T15:24:54.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.944+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31c0755c0 con 0x7fc31c0b2d60 2024-04-15T15:24:54.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.945+0000 7fc312a00640 1 -- 172.21.15.150:0/3910707811 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc30801a070 con 0x7fc31c0b2d60 2024-04-15T15:24:54.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.946+0000 7fc312a00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc2fc06c920 0x7fc2fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:54.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.946+0000 7fc312a00640 1 -- 172.21.15.150:0/3910707811 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5969+0+0 (secure 0 0 0) 0x7fc30808bb70 con 0x7fc31c0b2d60 2024-04-15T15:24:54.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.947+0000 7fc31a800640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc2fc06c920 0x7fc2fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:54.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.948+0000 7fc31a800640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc2fc06c920 0x7fc2fc06ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc30c036000 tx=0x7fc30c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:54.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:54.951+0000 7fc312a00640 1 -- 172.21.15.150:0/3910707811 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc31c0b2a60 con 0x7fc31c0b2d60 2024-04-15T15:24:55.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.191+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc31c06dbe0 con 0x7fc31c0b2d60 2024-04-15T15:24:55.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.723+0000 7fc312a00640 1 -- 172.21.15.150:0/3910707811 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fc308059df0 con 0x7fc31c0b2d60 2024-04-15T15:24:55.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.726+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc2fc06c920 msgr2=0x7fc2fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:55.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.726+0000 7fc320a00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc2fc06c920 0x7fc2fc06ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc30c036000 tx=0x7fc30c030000 comp rx=0 tx=0).stop 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.726+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31c0b2d60 msgr2=0x7fc31c15b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.726+0000 7fc320a00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31c0b2d60 0x7fc31c15b670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc30800e7a0 tx=0x7fc30800ec60 comp rx=0 tx=0).stop 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.727+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 shutdown_connections 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.727+0000 7fc320a00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc31c0c8b70 0x7fc31c160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.727+0000 7fc320a00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc31c0b2d60 0x7fc31c15b670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.727+0000 7fc320a00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc2fc06c920 0x7fc2fc06ede0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.727+0000 7fc320a00640 1 --2- 172.21.15.150:0/3910707811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3000039b0 0x7fc31c1f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.727+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 >> 172.21.15.150:0/3910707811 conn(0x7fc31c073480 msgr2=0x7fc31c0d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.727+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 shutdown_connections 2024-04-15T15:24:55.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.727+0000 7fc320a00640 1 -- 172.21.15.150:0/3910707811 wait complete. 2024-04-15T15:24:55.750 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-15T15:24:55.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.967+0000 7fd73be00640 1 Processor -- start 2024-04-15T15:24:55.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.967+0000 7fd73be00640 1 -- start start 2024-04-15T15:24:55.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.969+0000 7fd73be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd73c072620 0x7fd73c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.969+0000 7fd73be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd73c15d250 0x7fd73c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.969+0000 7fd73be00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c13fc70 con 0x7fd73c0cec60 2024-04-15T15:24:55.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.969+0000 7fd73be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c13f9e0 con 0x7fd73c15d250 2024-04-15T15:24:55.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.969+0000 7fd73be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c141a00 con 0x7fd73c072620 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.970+0000 7fd73aa00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd73c0cec60 0x7fd73c071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:56760/0 (socket says 172.21.15.150:56760) 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.970+0000 7fd73aa00640 1 -- 172.21.15.150:0/184335488 learned_addr learned my addr 172.21.15.150:0/184335488 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.970+0000 7fd73b400640 1 --2- 172.21.15.150:0/184335488 >> v2:172.21.15.187:3300/0 conn(0x7fd73c15d250 0x7fd73c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.970+0000 7fd732000640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd73c072620 0x7fd73c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.970+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193756696 0 0) 0x7fd73c13fc70 con 0x7fd73c0cec60 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.970+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd720003520 con 0x7fd73c0cec60 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2052052678 0 0) 0x7fd720003520 con 0x7fd73c0cec60 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd73c072620 msgr2=0x7fd73c15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd73c072620 0x7fd73c15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 >> v2:172.21.15.187:3300/0 conn(0x7fd73c15d250 msgr2=0x7fd73c06f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 --2- 172.21.15.150:0/184335488 >> v2:172.21.15.187:3300/0 conn(0x7fd73c15d250 0x7fd73c06f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c15da40 con 0x7fd73c0cec60 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fd72c0019f0 con 0x7fd73c0cec60 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 >> v1:172.21.15.150:6789/0 conn(0x7fd73c0cec60 legacy=0x7fd73c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200039b0 0x7fd720003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd73c072620 0x7fd720004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd720004d40 0x7fd7200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c141a00 con 0x7fd720004d40 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd72000fdd0 con 0x7fd7200039b0 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd72000ff50 con 0x7fd73c072620 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.971+0000 7fd73b400640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd720004d40 0x7fd7200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.972+0000 7fd732000640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd73c072620 0x7fd720004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.972+0000 7fd73aa00640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200039b0 0x7fd720003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.972+0000 7fd73b400640 1 -- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd73c072620 msgr2=0x7fd720004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.972+0000 7fd73b400640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd73c072620 0x7fd720004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.972+0000 7fd73b400640 1 -- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200039b0 msgr2=0x7fd720003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:55.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.972+0000 7fd73b400640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200039b0 0x7fd720003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.972+0000 7fd73b400640 1 -- 172.21.15.150:0/184335488 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd720003520 con 0x7fd720004d40 2024-04-15T15:24:55.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.972+0000 7fd73aa00640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7200039b0 0x7fd720003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:55.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.973+0000 7fd73b400640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd720004d40 0x7fd7200051c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd734009ac0 tx=0x7fd720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1182e4ff0e755e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:55.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.973+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734039070 con 0x7fd720004d40 2024-04-15T15:24:55.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.973+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73402c710 con 0x7fd720004d40 2024-04-15T15:24:55.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.973+0000 7fd73a000640 1 -- 172.21.15.150:0/184335488 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734009690 con 0x7fd720004d40 2024-04-15T15:24:55.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.974+0000 7fd73be00640 1 -- 172.21.15.150:0/184335488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd720004d40 msgr2=0x7fd7200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:55.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.974+0000 7fd73be00640 1 --2- 172.21.15.150:0/184335488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd720004d40 0x7fd7200051c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd734009ac0 tx=0x7fd720005830 comp rx=0 tx=0).stop 2024-04-15T15:24:55.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.974+0000 7fd73b400640 1 -- 172.21.15.150:0/184335488 reap_dead start 2024-04-15T15:24:55.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.974+0000 7fd73be00640 1 -- 172.21.15.150:0/184335488 shutdown_connections 2024-04-15T15:24:55.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.974+0000 7fd73be00640 1 -- 172.21.15.150:0/184335488 >> 172.21.15.150:0/184335488 conn(0x7fd73c0b6170 msgr2=0x7fd73c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:55.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.974+0000 7fd73be00640 1 -- 172.21.15.150:0/184335488 shutdown_connections 2024-04-15T15:24:55.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.974+0000 7fd73be00640 1 -- 172.21.15.150:0/184335488 wait complete. 2024-04-15T15:24:55.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.976+0000 7fd73be00640 1 Processor -- start 2024-04-15T15:24:55.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.976+0000 7fd73be00640 1 -- start start 2024-04-15T15:24:55.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd720004d40 0x7fd73c0abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd73c0ac1f0 0x7fd73c1f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd73c1f37a0 0x7fd73c13d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1404d0 con 0x7fd73c1f37a0 2024-04-15T15:24:55.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1401f0 con 0x7fd73c0ac1f0 2024-04-15T15:24:55.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c15d700 con 0x7fd720004d40 2024-04-15T15:24:55.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd73c1f37a0 0x7fd73c13d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd73c1f37a0 0x7fd73c13d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33806/0 (socket says 172.21.15.150:33806) 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.977+0000 7fd73b400640 1 -- 172.21.15.150:0/2676119952 learned_addr learned my addr 172.21.15.150:0/2676119952 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd73b400640 1 -- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd720004d40 msgr2=0x7fd73c0abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd73aa00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd720004d40 0x7fd73c0abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd732000640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd73c0ac1f0 0x7fd73c1f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd73b400640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd720004d40 0x7fd73c0abcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd73b400640 1 -- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd73c0ac1f0 msgr2=0x7fd73c1f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd73b400640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd73c0ac1f0 0x7fd73c1f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd73b400640 1 -- 172.21.15.150:0/2676119952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c0b2150 con 0x7fd73c1f37a0 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd73b400640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd73c1f37a0 0x7fd73c13d570 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd73402c380 tx=0x7fd734004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd733400640 1 -- 172.21.15.150:0/2676119952 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734039070 con 0x7fd73c1f37a0 2024-04-15T15:24:55.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd733400640 1 -- 172.21.15.150:0/2676119952 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd734043070 con 0x7fd73c1f37a0 2024-04-15T15:24:55.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd733400640 1 -- 172.21.15.150:0/2676119952 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734009690 con 0x7fd73c1f37a0 2024-04-15T15:24:55.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.978+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c0ae7a0 con 0x7fd73c1f37a0 2024-04-15T15:24:55.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.979+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c0acb00 con 0x7fd73c1f37a0 2024-04-15T15:24:55.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.980+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c068b40 con 0x7fd73c1f37a0 2024-04-15T15:24:55.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.981+0000 7fd733400640 1 -- 172.21.15.150:0/2676119952 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd734003a30 con 0x7fd73c1f37a0 2024-04-15T15:24:55.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.981+0000 7fd733400640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd71806c8f0 0x7fd71806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:55.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.982+0000 7fd733400640 1 -- 172.21.15.150:0/2676119952 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5969+0+0 (secure 0 0 0) 0x7fd7340acd40 con 0x7fd73c1f37a0 2024-04-15T15:24:55.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.982+0000 7fd73aa00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd71806c8f0 0x7fd71806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:55.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.983+0000 7fd73aa00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd71806c8f0 0x7fd71806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00e280 tx=0x7fd72c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:55.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:55.986+0000 7fd733400640 1 -- 172.21.15.150:0/2676119952 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd73404c060 con 0x7fd73c1f37a0 2024-04-15T15:24:56.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.187+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd73c0730e0 con 0x7fd73c1f37a0 2024-04-15T15:24:56.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.727+0000 7fd733400640 1 -- 172.21.15.150:0/2676119952 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7fd73407b050 con 0x7fd73c1f37a0 2024-04-15T15:24:56.729 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_metadata' removed 2024-04-15T15:24:56.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd71806c8f0 msgr2=0x7fd71806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:56.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd71806c8f0 0x7fd71806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00e280 tx=0x7fd72c009040 comp rx=0 tx=0).stop 2024-04-15T15:24:56.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd73c1f37a0 msgr2=0x7fd73c13d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:56.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd73c1f37a0 0x7fd73c13d570 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd73402c380 tx=0x7fd734004730 comp rx=0 tx=0).stop 2024-04-15T15:24:56.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 shutdown_connections 2024-04-15T15:24:56.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd73c1f37a0 0x7fd73c13d570 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:56.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd73c0ac1f0 0x7fd73c1f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:56.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd71806c8f0 0x7fd71806edb0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00e280 tx=0x7fd72c009040 comp rx=0 tx=0).stop 2024-04-15T15:24:56.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 --2- 172.21.15.150:0/2676119952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd720004d40 0x7fd73c0abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:56.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.730+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 >> 172.21.15.150:0/2676119952 conn(0x7fd73c0b6170 msgr2=0x7fd73c15f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:56.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.731+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 shutdown_connections 2024-04-15T15:24:56.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.731+0000 7fd73be00640 1 -- 172.21.15.150:0/2676119952 wait complete. 2024-04-15T15:24:56.768 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-15T15:24:56.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.959+0000 7f0ca8200640 1 Processor -- start 2024-04-15T15:24:56.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.959+0000 7f0ca8200640 1 -- start start 2024-04-15T15:24:56.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.961+0000 7f0ca8200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ca0072860 0x7f0ca0169620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:56.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.961+0000 7f0ca8200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca01454c0 con 0x7f0ca00af490 2024-04-15T15:24:56.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.961+0000 7f0ca8200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca00af920 con 0x7f0ca01450e0 2024-04-15T15:24:56.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.961+0000 7f0ca8200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca00b01c0 con 0x7f0ca0072860 2024-04-15T15:24:56.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.961+0000 7f0ca6200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ca0072860 0x7f0ca0169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:56.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.961+0000 7f0ca4e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0ca00af490 0x7f0ca0072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50882/0 (socket says 172.21.15.150:50882) 2024-04-15T15:24:56.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.961+0000 7f0ca4e00640 1 -- 172.21.15.150:0/1516505885 learned_addr learned my addr 172.21.15.150:0/1516505885 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:56.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.962+0000 7f0ca6200640 1 -- 172.21.15.150:0/1516505885 >> v1:172.21.15.187:6789/0 conn(0x7f0ca01450e0 legacy=0x7f0ca00aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:56.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.962+0000 7f0ca6200640 1 -- 172.21.15.150:0/1516505885 >> v1:172.21.15.150:6789/0 conn(0x7f0ca00af490 legacy=0x7f0ca0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:56.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.962+0000 7f0c9fe00640 1 -- 172.21.15.150:0/1516505885 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025225512 0 0) 0x7f0ca01454c0 con 0x7f0ca00af490 2024-04-15T15:24:56.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.962+0000 7f0ca6200640 1 -- 172.21.15.150:0/1516505885 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca00afc00 con 0x7f0ca0072860 2024-04-15T15:24:56.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.963+0000 7f0ca6200640 1 --2- 172.21.15.150:0/1516505885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca0072860 0x7f0ca0169620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009d90 tx=0x7f0c9802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1b98cc1f2be3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:56.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.963+0000 7f0c9fe00640 1 -- 172.21.15.150:0/1516505885 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98039070 con 0x7f0ca0072860 2024-04-15T15:24:56.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0c9fe00640 1 -- 172.21.15.150:0/1516505885 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9802cc60 con 0x7f0ca0072860 2024-04-15T15:24:56.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0c9fe00640 1 -- 172.21.15.150:0/1516505885 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98034660 con 0x7f0ca0072860 2024-04-15T15:24:56.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0ca8200640 1 -- 172.21.15.150:0/1516505885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca0072860 msgr2=0x7f0ca0169620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:56.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0ca8200640 1 --2- 172.21.15.150:0/1516505885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca0072860 0x7f0ca0169620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009d90 tx=0x7f0c9802beb0 comp rx=0 tx=0).stop 2024-04-15T15:24:56.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0ca8200640 1 -- 172.21.15.150:0/1516505885 shutdown_connections 2024-04-15T15:24:56.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0ca8200640 1 --2- 172.21.15.150:0/1516505885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca0072860 0x7f0ca0169620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:56.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0ca8200640 1 -- 172.21.15.150:0/1516505885 >> 172.21.15.150:0/1516505885 conn(0x7f0ca00b2460 msgr2=0x7f0ca00b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:56.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0ca8200640 1 -- 172.21.15.150:0/1516505885 shutdown_connections 2024-04-15T15:24:56.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.964+0000 7f0ca8200640 1 -- 172.21.15.150:0/1516505885 wait complete. 2024-04-15T15:24:56.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.966+0000 7f0ca8200640 1 Processor -- start 2024-04-15T15:24:56.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.967+0000 7f0ca8200640 1 -- start start 2024-04-15T15:24:56.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ca0072860 0x7f0ca015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:56.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca8200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ca01450e0 0x7f0ca01e4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:56.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca8200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca01e4b50 0x7f0ca01e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:56.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca8200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca01484b0 con 0x7f0ca01e4b50 2024-04-15T15:24:56.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca8200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca00b3890 con 0x7f0ca01450e0 2024-04-15T15:24:56.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca0148630 con 0x7f0ca0072860 2024-04-15T15:24:56.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca6200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca01e4b50 0x7f0ca01e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:56.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca6200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca01e4b50 0x7f0ca01e8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53094/0 (socket says 172.21.15.150:53094) 2024-04-15T15:24:56.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca6200640 1 -- 172.21.15.150:0/2866629465 learned_addr learned my addr 172.21.15.150:0/2866629465 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca5800640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ca0072860 0x7f0ca015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.968+0000 7f0ca4e00640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ca01450e0 0x7f0ca01e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca6200640 1 -- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ca0072860 msgr2=0x7f0ca015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca6200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ca0072860 0x7f0ca015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca6200640 1 -- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ca01450e0 msgr2=0x7f0ca01e4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca6200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ca01450e0 0x7f0ca01e4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca6200640 1 -- 172.21.15.150:0/2866629465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca01e63c0 con 0x7f0ca01e4b50 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca4e00640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ca01450e0 0x7f0ca01e4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca6200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca01e4b50 0x7f0ca01e8ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0c98037040 tx=0x7f0c98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0c9d600640 1 -- 172.21.15.150:0/2866629465 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98044020 con 0x7f0ca01e4b50 2024-04-15T15:24:56.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0c9d600640 1 -- 172.21.15.150:0/2866629465 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9803a040 con 0x7f0ca01e4b50 2024-04-15T15:24:56.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca01e66a0 con 0x7f0ca01e4b50 2024-04-15T15:24:56.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0c9d600640 1 -- 172.21.15.150:0/2866629465 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98039050 con 0x7f0ca01e4b50 2024-04-15T15:24:56.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.969+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca01e6c60 con 0x7f0ca01e4b50 2024-04-15T15:24:56.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.970+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca0144c40 con 0x7f0ca01e4b50 2024-04-15T15:24:56.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.972+0000 7f0c9d600640 1 -- 172.21.15.150:0/2866629465 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c9802cae0 con 0x7f0ca01e4b50 2024-04-15T15:24:56.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.972+0000 7f0c9d600640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0c8006c8f0 0x7f0c8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:56.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.973+0000 7f0c9d600640 1 -- 172.21.15.150:0/2866629465 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5531+0+0 (secure 0 0 0) 0x7f0c980ae200 con 0x7f0ca01e4b50 2024-04-15T15:24:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.973+0000 7f0ca5800640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0c8006c8f0 0x7f0c8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:56.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.974+0000 7f0ca5800640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0c8006c8f0 0x7f0c8006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0143e00 tx=0x7f0c94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:56.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:56.979+0000 7f0c9d600640 1 -- 172.21.15.150:0/2866629465 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ca0144c40 con 0x7f0ca01e4b50 2024-04-15T15:24:57.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.191+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ca0068db0 con 0x7f0ca01e4b50 2024-04-15T15:24:57.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.730+0000 7f0c9d600640 1 -- 172.21.15.150:0/2866629465 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7f0c9804e060 con 0x7f0ca01e4b50 2024-04-15T15:24:57.732 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data' removed 2024-04-15T15:24:57.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.733+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0c8006c8f0 msgr2=0x7f0c8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:57.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.733+0000 7f0ca8200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0c8006c8f0 0x7f0c8006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0143e00 tx=0x7f0c94009040 comp rx=0 tx=0).stop 2024-04-15T15:24:57.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.733+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca01e4b50 msgr2=0x7f0ca01e8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:57.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.733+0000 7f0ca8200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca01e4b50 0x7f0ca01e8ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0c98037040 tx=0x7f0c98004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:57.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.734+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 shutdown_connections 2024-04-15T15:24:57.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.734+0000 7f0ca8200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca01e4b50 0x7f0ca01e8ec0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:57.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.734+0000 7f0ca8200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ca01450e0 0x7f0ca01e4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:57.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.734+0000 7f0ca8200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0c8006c8f0 0x7f0c8006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:57.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.734+0000 7f0ca8200640 1 --2- 172.21.15.150:0/2866629465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ca0072860 0x7f0ca015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:57.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.734+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 >> 172.21.15.150:0/2866629465 conn(0x7f0ca00b2460 msgr2=0x7f0ca01457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:57.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.734+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 shutdown_connections 2024-04-15T15:24:57.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:57.734+0000 7f0ca8200640 1 -- 172.21.15.150:0/2866629465 wait complete. 2024-04-15T15:24:57.766 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-15T15:24:57.766 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-15T15:24:57.766 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-15T15:24:57.766 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:24:57.767 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-15T15:24:57.767 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-15T15:24:57.767 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-15T15:24:57.767 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:24:57.767 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-15T15:24:57.767 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-15T15:24:57.768 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-15T15:24:57.768 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:24:57.768 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-15T15:24:57.768 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-15T15:24:57.768 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-15T15:24:57.768 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:24:57.788 INFO:tasks.ceph.mds.a:Stopped 2024-04-15T15:24:57.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-15T15:24:57.789 INFO:tasks.ceph.mds.d:Stopped 2024-04-15T15:24:57.789 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-15T15:24:57.792 INFO:tasks.ceph.mds.c:Stopped 2024-04-15T15:24:57.792 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-15T15:24:57.795 INFO:tasks.ceph.mds.b:Stopped 2024-04-15T15:24:57.795 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-15T15:24:57.836 INFO:tasks.ceph.mds.a:Started 2024-04-15T15:24:57.837 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:57.840 INFO:tasks.ceph.mds.c:Started 2024-04-15T15:24:57.840 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:57.842 INFO:tasks.ceph.mds.d:Started 2024-04-15T15:24:57.842 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:57.848 INFO:tasks.ceph.mds.b:Started 2024-04-15T15:24:57.848 DEBUG:teuthology.parallel:result is None 2024-04-15T15:24:57.848 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-15T15:24:57.994 INFO:tasks.ceph.mds.c.smithi150.stdout:starting mds.c at 2024-04-15T15:24:57.994 INFO:tasks.ceph.mds.a.smithi150.stdout:starting mds.a at 2024-04-15T15:24:58.006 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-04-15T15:24:58.007 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-15T15:24:58.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.040+0000 7fa9c8200640 1 Processor -- start 2024-04-15T15:24:58.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.040+0000 7fa9c8200640 1 -- start start 2024-04-15T15:24:58.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.041+0000 7fa9c8200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa9c0162e90 0x7fa9c00b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:58.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.042+0000 7fa9c8200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9c00b01d0 con 0x7fa9c0072810 2024-04-15T15:24:58.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.042+0000 7fa9c8200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9c00b3900 con 0x7fa9c00afd70 2024-04-15T15:24:58.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.042+0000 7fa9c8200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa9c00b3be0 con 0x7fa9c0162e90 2024-04-15T15:24:58.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.042+0000 7fa9c6200640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa9c0072810 0x7fa9c00ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50904/0 (socket says 172.21.15.150:50904) 2024-04-15T15:24:58.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.042+0000 7fa9c6200640 1 -- 172.21.15.150:0/652818062 learned_addr learned my addr 172.21.15.150:0/652818062 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:58.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.042+0000 7fa9c5800640 1 --2- 172.21.15.150:0/652818062 >> v2:172.21.15.187:3300/0 conn(0x7fa9c0162e90 0x7fa9c00b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:58.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.043+0000 7fa9bfe00640 1 -- 172.21.15.150:0/652818062 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094397828 0 0) 0x7fa9b8001490 con 0x7fa9c0072810 2024-04-15T15:24:58.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.043+0000 7fa9bfe00640 1 -- 172.21.15.150:0/652818062 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9a8003520 con 0x7fa9c0072810 2024-04-15T15:24:58.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.043+0000 7fa9c5800640 1 -- 172.21.15.150:0/652818062 >> v1:172.21.15.187:6789/0 conn(0x7fa9c00afd70 legacy=0x7fa9c0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:58.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.043+0000 7fa9bfe00640 1 -- 172.21.15.150:0/652818062 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3198364000 0 0) 0x7fa9c00b3900 con 0x7fa9c00afd70 2024-04-15T15:24:58.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.043+0000 7fa9c5800640 1 -- 172.21.15.150:0/652818062 >> v1:172.21.15.150:6789/0 conn(0x7fa9c0072810 legacy=0x7fa9c00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:58.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.043+0000 7fa9c5800640 1 -- 172.21.15.150:0/652818062 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c00b3ec0 con 0x7fa9c0162e90 2024-04-15T15:24:58.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.044+0000 7fa9c5800640 1 --2- 172.21.15.150:0/652818062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 0x7fa9c00b58e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40097a0 tx=0x7fa9b402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54ef3fc0fafe622 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:58.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.044+0000 7fa9bfe00640 1 -- 172.21.15.150:0/652818062 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4035020 con 0x7fa9c0162e90 2024-04-15T15:24:58.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.044+0000 7fa9bfe00640 1 -- 172.21.15.150:0/652818062 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b4031830 con 0x7fa9c0162e90 2024-04-15T15:24:58.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.044+0000 7fa9bfe00640 1 -- 172.21.15.150:0/652818062 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4031c00 con 0x7fa9c0162e90 2024-04-15T15:24:58.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.045+0000 7fa9c8200640 1 -- 172.21.15.150:0/652818062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 msgr2=0x7fa9c00b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:58.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.045+0000 7fa9c8200640 1 --2- 172.21.15.150:0/652818062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 0x7fa9c00b58e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40097a0 tx=0x7fa9b402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:24:58.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.045+0000 7fa9c8200640 1 -- 172.21.15.150:0/652818062 shutdown_connections 2024-04-15T15:24:58.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.045+0000 7fa9c8200640 1 --2- 172.21.15.150:0/652818062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 0x7fa9c00b58e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:58.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.045+0000 7fa9c8200640 1 -- 172.21.15.150:0/652818062 >> 172.21.15.150:0/652818062 conn(0x7fa9c00b2460 msgr2=0x7fa9c00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:58.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.045+0000 7fa9c8200640 1 -- 172.21.15.150:0/652818062 shutdown_connections 2024-04-15T15:24:58.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.045+0000 7fa9c8200640 1 -- 172.21.15.150:0/652818062 wait complete. 2024-04-15T15:24:58.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.047+0000 7fa9c8200640 1 Processor -- start 2024-04-15T15:24:58.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.047+0000 7fa9c8200640 1 -- start start 2024-04-15T15:24:58.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c0072810 0x7fa9c01508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:58.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c8200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9c00afd70 0x7fa9c015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:58.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c8200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 0x7fa9c01f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:58.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c8200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c00cf020 con 0x7fa9c00afd70 2024-04-15T15:24:58.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c8200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c00ceea0 con 0x7fa9c0162e90 2024-04-15T15:24:58.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c00cf1a0 con 0x7fa9c0072810 2024-04-15T15:24:58.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c4e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9c00afd70 0x7fa9c015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:58.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c4e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9c00afd70 0x7fa9c015a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53146/0 (socket says 172.21.15.150:53146) 2024-04-15T15:24:58.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c4e00640 1 -- 172.21.15.150:0/3207852389 learned_addr learned my addr 172.21.15.150:0/3207852389 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:58.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c5800640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c0072810 0x7fa9c01508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:58.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.048+0000 7fa9c6200640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 0x7fa9c01f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:58.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9c4e00640 1 -- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c0072810 msgr2=0x7fa9c01508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:58.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9c4e00640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c0072810 0x7fa9c01508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:58.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9c4e00640 1 -- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 msgr2=0x7fa9c01f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:58.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9c4e00640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 0x7fa9c01f8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:58.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9c4e00640 1 -- 172.21.15.150:0/3207852389 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c015ae50 con 0x7fa9c00afd70 2024-04-15T15:24:58.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9c4e00640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9c00afd70 0x7fa9c015a470 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00c4b0 tx=0x7fa9ac00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:58.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9bd600640 1 -- 172.21.15.150:0/3207852389 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac017070 con 0x7fa9c00afd70 2024-04-15T15:24:58.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9bd600640 1 -- 172.21.15.150:0/3207852389 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ac005480 con 0x7fa9c00afd70 2024-04-15T15:24:58.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9bd600640 1 -- 172.21.15.150:0/3207852389 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac003830 con 0x7fa9c00afd70 2024-04-15T15:24:58.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.049+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c015b130 con 0x7fa9c00afd70 2024-04-15T15:24:58.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.050+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c015b6f0 con 0x7fa9c00afd70 2024-04-15T15:24:58.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.051+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c0068c80 con 0x7fa9c00afd70 2024-04-15T15:24:58.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.052+0000 7fa9bd600640 1 -- 172.21.15.150:0/3207852389 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9ac01a070 con 0x7fa9c00afd70 2024-04-15T15:24:58.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.052+0000 7fa9bd600640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9a006c8f0 0x7fa9a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:58.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.053+0000 7fa9bd600640 1 -- 172.21.15.150:0/3207852389 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7fa9ac08ab40 con 0x7fa9c00afd70 2024-04-15T15:24:58.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.053+0000 7fa9c5800640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9a006c8f0 0x7fa9a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:58.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.054+0000 7fa9c5800640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9a006c8f0 0x7fa9a006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4036000 tx=0x7fa9b403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:58.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.057+0000 7fa9bd600640 1 -- 172.21.15.150:0/3207852389 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9ac058c20 con 0x7fa9c00afd70 2024-04-15T15:24:58.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.269+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa9c006c670 con 0x7fa9c00afd70 2024-04-15T15:24:58.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.769+0000 7fa9bd600640 1 -- 172.21.15.150:0/3207852389 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7fa9ac05cf20 con 0x7fa9c00afd70 2024-04-15T15:24:58.772 INFO:teuthology.orchestra.run.smithi150.stderr: removed all blocklist entries 2024-04-15T15:24:58.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9a006c8f0 msgr2=0x7fa9a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:58.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9a006c8f0 0x7fa9a006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4036000 tx=0x7fa9b403c040 comp rx=0 tx=0).stop 2024-04-15T15:24:58.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9c00afd70 msgr2=0x7fa9c015a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:58.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9c00afd70 0x7fa9c015a470 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00c4b0 tx=0x7fa9ac00c970 comp rx=0 tx=0).stop 2024-04-15T15:24:58.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 shutdown_connections 2024-04-15T15:24:58.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c0162e90 0x7fa9c01f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:58.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9c00afd70 0x7fa9c015a470 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:58.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9a006c8f0 0x7fa9a006edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:58.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 --2- 172.21.15.150:0/3207852389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c0072810 0x7fa9c01508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:58.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.773+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 >> 172.21.15.150:0/3207852389 conn(0x7fa9c00b2460 msgr2=0x7fa9c006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:58.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.774+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 shutdown_connections 2024-04-15T15:24:58.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:58.774+0000 7fa9c8200640 1 -- 172.21.15.150:0/3207852389 wait complete. 2024-04-15T15:24:58.799 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-15T15:24:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.000+0000 7f65ee000640 1 Processor -- start 2024-04-15T15:24:59.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.000+0000 7f65ee000640 1 -- start start 2024-04-15T15:24:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.001+0000 7f65ee000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 0x7f65e80b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.001+0000 7f65ee000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f65e80b5020 0x7f65e815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.001+0000 7f65ee000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e806f940 con 0x7f65e815b6d0 2024-04-15T15:24:59.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.001+0000 7f65ee000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f65e80b5480 con 0x7f65e80b5020 2024-04-15T15:24:59.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.001+0000 7f65ee000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65e806e8a0 con 0x7f65e8070210 2024-04-15T15:24:59.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.002+0000 7f65e7e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f65e80b5020 0x7f65e815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.002+0000 7f65e7e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f65e80b5020 0x7f65e815bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53150/0 (socket says 172.21.15.150:53150) 2024-04-15T15:24:59.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.002+0000 7f65e7e00640 1 -- 172.21.15.150:0/2106620355 learned_addr learned my addr 172.21.15.150:0/2106620355 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.002+0000 7f65dfe00640 1 --2- 172.21.15.150:0/2106620355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 0x7f65e80b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.002+0000 7f65e7e00640 1 -- 172.21.15.150:0/2106620355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 msgr2=0x7f65e80b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.002+0000 7f65e7e00640 1 --2- 172.21.15.150:0/2106620355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 0x7f65e80b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.002+0000 7f65e7e00640 1 -- 172.21.15.150:0/2106620355 >> v1:172.21.15.187:6789/0 conn(0x7f65e815b6d0 legacy=0x7f65e80cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.002+0000 7f65e7e00640 1 -- 172.21.15.150:0/2106620355 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e806eb80 con 0x7f65e80b5020 2024-04-15T15:24:59.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.003+0000 7f65e7e00640 1 --2- 172.21.15.150:0/2106620355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e80b5020 0x7f65e815bab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009e50 tx=0x7f65e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9a19384b74f9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65e6a00640 1 -- 172.21.15.150:0/2106620355 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e002c9e0 con 0x7f65e80b5020 2024-04-15T15:24:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65e6a00640 1 -- 172.21.15.150:0/2106620355 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e002cb80 con 0x7f65e80b5020 2024-04-15T15:24:59.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65e6a00640 1 -- 172.21.15.150:0/2106620355 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e00343f0 con 0x7f65e80b5020 2024-04-15T15:24:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65ee000640 1 -- 172.21.15.150:0/2106620355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e80b5020 msgr2=0x7f65e815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65ee000640 1 --2- 172.21.15.150:0/2106620355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e80b5020 0x7f65e815bab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009e50 tx=0x7f65e002be40 comp rx=0 tx=0).stop 2024-04-15T15:24:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65ee000640 1 -- 172.21.15.150:0/2106620355 shutdown_connections 2024-04-15T15:24:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65ee000640 1 --2- 172.21.15.150:0/2106620355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e80b5020 0x7f65e815bab0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65ee000640 1 --2- 172.21.15.150:0/2106620355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 0x7f65e80b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65ee000640 1 -- 172.21.15.150:0/2106620355 >> 172.21.15.150:0/2106620355 conn(0x7f65e80b27b0 msgr2=0x7f65e80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65ee000640 1 -- 172.21.15.150:0/2106620355 shutdown_connections 2024-04-15T15:24:59.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.004+0000 7f65ee000640 1 -- 172.21.15.150:0/2106620355 wait complete. 2024-04-15T15:24:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.006+0000 7f65ee000640 1 Processor -- start 2024-04-15T15:24:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.006+0000 7f65ee000640 1 -- start start 2024-04-15T15:24:59.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.007+0000 7f65ee000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 0x7f65e8143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.007+0000 7f65ee000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65e80b5020 0x7f65e80ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.007+0000 7f65ee000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e815b6d0 0x7f65e81e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.007+0000 7f65ee000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65e806f420 con 0x7f65e815b6d0 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.007+0000 7f65ee000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65e806f140 con 0x7f65e80b5020 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.007+0000 7f65ee000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65e80ce420 con 0x7f65e8070210 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.007+0000 7f65e7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e815b6d0 0x7f65e81e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e815b6d0 0x7f65e81e39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53158/0 (socket says 172.21.15.150:53158) 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7e00640 1 -- 172.21.15.150:0/753448691 learned_addr learned my addr 172.21.15.150:0/753448691 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65dfe00640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65e80b5020 0x7f65e80ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7400640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 0x7f65e8143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7e00640 1 -- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 msgr2=0x7f65e8143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7e00640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 0x7f65e8143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7e00640 1 -- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65e80b5020 msgr2=0x7f65e80ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7e00640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65e80b5020 0x7f65e80ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7e00640 1 -- 172.21.15.150:0/753448691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e81443c0 con 0x7f65e815b6d0 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65dfe00640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65e80b5020 0x7f65e80ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.008+0000 7f65e7e00640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e815b6d0 0x7f65e81e39c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f65e0036040 tx=0x7f65e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.009+0000 7f65df400640 1 -- 172.21.15.150:0/753448691 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0043020 con 0x7f65e815b6d0 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.009+0000 7f65df400640 1 -- 172.21.15.150:0/753448691 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e003e450 con 0x7f65e815b6d0 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.009+0000 7f65df400640 1 -- 172.21.15.150:0/753448691 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e003e860 con 0x7f65e815b6d0 2024-04-15T15:24:59.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.009+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e8144a90 con 0x7f65e815b6d0 2024-04-15T15:24:59.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.009+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e8145050 con 0x7f65e815b6d0 2024-04-15T15:24:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.010+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e8068c80 con 0x7f65e815b6d0 2024-04-15T15:24:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.011+0000 7f65df400640 1 -- 172.21.15.150:0/753448691 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65e0046020 con 0x7f65e815b6d0 2024-04-15T15:24:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.012+0000 7f65df400640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f65c406c8f0 0x7f65c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.012+0000 7f65df400640 1 -- 172.21.15.150:0/753448691 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f65e00ae1f0 con 0x7f65e815b6d0 2024-04-15T15:24:59.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.013+0000 7f65e7400640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f65c406c8f0 0x7f65c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.014+0000 7f65e7400640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f65c406c8f0 0x7f65c406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65d800a750 tx=0x7f65d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:59.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.017+0000 7f65df400640 1 -- 172.21.15.150:0/753448691 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65e004d030 con 0x7f65e815b6d0 2024-04-15T15:24:59.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.289+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f65e80b4d70 con 0x7f65e815b6d0 2024-04-15T15:24:59.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.289+0000 7f65df400640 1 -- 172.21.15.150:0/753448691 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f65e007c8d0 con 0x7f65e815b6d0 2024-04-15T15:24:59.291 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_dump": [ 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "mds.a", 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBKRh1mpj3+HRAAEh8ivBX0TX5YWjk24rgdsQ==", 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow", 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow *", 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow *", 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.292 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "mds.b", 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBLRh1mjOuJBBAAeGWYhd6bfV40OMnCvrcapQ==", 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow", 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow *", 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow *", 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "mds.c", 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBKRh1mLyTjLhAA932wttbBinyuX0HaTQs1iA==", 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow", 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow *", 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow *", 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "mds.d", 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBLRh1mrwjUFBAAtMnGVZa/OdsIXd1N/dI2TQ==", 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow", 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow *", 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow *", 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "osd.0", 2024-04-15T15:24:59.295 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBPRh1mhJRsIRAAzN77WIEMVi/OrEPGuj+3Zg==", 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow profile osd", 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile osd", 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "osd.1", 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBSRh1mZmF4LhAA2ikl51iHw8oUr3U2kXIVZQ==", 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.296 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow profile osd", 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile osd", 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "osd.2", 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBVRh1mzaaFOhAAUeYLxRNUQOdbZsE7D4GYjQ==", 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow profile osd", 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile osd", 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.297 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "osd.3", 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBZRh1mLMEsCRAADKns6IJ7E6eHoGjlGRRZ1A==", 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow profile osd", 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile osd", 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "osd.4", 2024-04-15T15:24:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBfRh1my6zIJRAAJb2eCGYAs/3ny43n+A/Z6w==", 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow profile osd", 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile osd", 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "osd.5", 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBiRh1mwPS/MhAAy7mjMXTIoDoULw6U9E+z8Q==", 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow profile osd", 2024-04-15T15:24:59.299 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile osd", 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "osd.6", 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBmRh1mCKFbAhAAPRINFqJje8M4SfTJe13S0A==", 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow profile osd", 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile osd", 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "osd.7", 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBpRh1maamQDhAAiD+0Z5GY3lCtdDZCQEknBA==", 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow profile osd", 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile osd", 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.0", 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBLRh1mMhWcIhAALcn3OkkYUWxiZFNe4T1a0A==", 2024-04-15T15:24:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow", 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow r", 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow rw", 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow rwx" 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.1", 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBLRh1md4XlOBAAVojA1/Gr3hDMDnqtV1d2GA==", 2024-04-15T15:24:59.302 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow", 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow r", 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow rw", 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow rwx" 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.2", 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBLRh1mfVuJLRAAMLqOcfYptvjLOA6WhIto/g==", 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow", 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow r", 2024-04-15T15:24:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow rw", 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow rwx" 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.3", 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBMRh1mA5+cBxAA+C06nPelPPXzUGWAGHi+0w==", 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow", 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow r", 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow rw", 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow rwx" 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.admin", 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBJRh1mzs/JExAAMRAEJgi8Gubfs+PV71UwXA==", 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow *", 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": "allow *", 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow *", 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.bootstrap-mds", 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQByRh1mWRTVBRAAU+/BeeJKymEiIax4yhvbuQ==", 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile bootstrap-mds" 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.bootstrap-mgr", 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQByRh1mAyTVBRAAQ/g6Lte28C/oQK17jmLHqw==", 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-15T15:24:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.bootstrap-osd", 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQByRh1m4DDVBRAA7j57ybbuA1KoV8PZ49m/hg==", 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile bootstrap-osd" 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.bootstrap-rbd", 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQByRh1mMj3VBRAAh18H7cMwuZ/CTNl/HR8/Qg==", 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-15T15:24:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQByRh1mAkrVBRAA8eyfW1bI9yXCecbEfeDEuQ==", 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "client.bootstrap-rgw", 2024-04-15T15:24:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQByRh1mPlnVBRAAfDcoxBEBFPiYuDSAeStY0Q==", 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "mgr.x", 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBKRh1mFIJlEhAA43SJvZPpyghWYT16hxEnIw==", 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow *", 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile mgr", 2024-04-15T15:24:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": "mgr.y", 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "AQBKRh1md7UyBhAAlb270hfgvbw7PyNlE4iJyQ==", 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": { 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": "allow *", 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "mon": "allow profile mgr", 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": "allow *" 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:24:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-04-15T15:24:59.311 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:24:59.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f65c406c8f0 msgr2=0x7f65c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f65c406c8f0 0x7f65c406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65d800a750 tx=0x7f65d8008040 comp rx=0 tx=0).stop 2024-04-15T15:24:59.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e815b6d0 msgr2=0x7f65e81e39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e815b6d0 0x7f65e81e39c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f65e0036040 tx=0x7f65e0004a40 comp rx=0 tx=0).stop 2024-04-15T15:24:59.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 shutdown_connections 2024-04-15T15:24:59.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e815b6d0 0x7f65e81e39c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65e80b5020 0x7f65e80ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f65c406c8f0 0x7f65c406edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 --2- 172.21.15.150:0/753448691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65e8070210 0x7f65e8143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 >> 172.21.15.150:0/753448691 conn(0x7f65e80b27b0 msgr2=0x7f65e80cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:59.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 shutdown_connections 2024-04-15T15:24:59.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.292+0000 7f65ee000640 1 -- 172.21.15.150:0/753448691 wait complete. 2024-04-15T15:24:59.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-15T15:24:59.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.522+0000 7ffb6ec00640 1 Processor -- start 2024-04-15T15:24:59.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.523+0000 7ffb6ec00640 1 -- start start 2024-04-15T15:24:59.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.524+0000 7ffb6ec00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffb680cf9b0 0x7ffb68163090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.524+0000 7ffb6ec00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb680d2d20 con 0x7ffb680d48a0 2024-04-15T15:24:59.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.524+0000 7ffb6ec00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb680cff70 con 0x7ffb68161fc0 2024-04-15T15:24:59.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.524+0000 7ffb6ec00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ffb680d2fc0 con 0x7ffb680cf9b0 2024-04-15T15:24:59.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.524+0000 7ffb6cc00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7ffb680d48a0 0x7ffb680d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50906/0 (socket says 172.21.15.150:50906) 2024-04-15T15:24:59.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.524+0000 7ffb6cc00640 1 -- 172.21.15.150:0/3605946776 learned_addr learned my addr 172.21.15.150:0/3605946776 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb67400640 1 --2- 172.21.15.150:0/3605946776 >> v2:172.21.15.187:3300/0 conn(0x7ffb680cf9b0 0x7ffb68163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2597409696 0 0) 0x7ffb680d2d20 con 0x7ffb680d48a0 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb4c003520 con 0x7ffb680d48a0 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 526876979 0 0) 0x7ffb4c003520 con 0x7ffb680d48a0 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 >> v2:172.21.15.187:3300/0 conn(0x7ffb680cf9b0 msgr2=0x7ffb68163090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 --2- 172.21.15.150:0/3605946776 >> v2:172.21.15.187:3300/0 conn(0x7ffb680cf9b0 0x7ffb68163090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 >> v1:172.21.15.187:6789/0 conn(0x7ffb68161fc0 legacy=0x7ffb680b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb680d32a0 con 0x7ffb680d48a0 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7ffb60001670 con 0x7ffb680d48a0 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 >> v1:172.21.15.150:6789/0 conn(0x7ffb680d48a0 legacy=0x7ffb680d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb4c003a00 0x7ffb4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c004800 0x7ffb4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb4c0051c0 0x7ffb4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb680d2fc0 con 0x7ffb4c0051c0 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c00c090 con 0x7ffb4c004800 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.525+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c00c290 con 0x7ffb4c003a00 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.526+0000 7ffb6cc00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb4c0051c0 0x7ffb4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.526+0000 7ffb67400640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c004800 0x7ffb4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.526+0000 7ffb67e00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb4c003a00 0x7ffb4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.526+0000 7ffb6cc00640 1 -- 172.21.15.150:0/3605946776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb4c003a00 msgr2=0x7ffb4c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.526+0000 7ffb6cc00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb4c003a00 0x7ffb4c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.526+0000 7ffb6cc00640 1 -- 172.21.15.150:0/3605946776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c004800 msgr2=0x7ffb4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.526+0000 7ffb6cc00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c004800 0x7ffb4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.526+0000 7ffb6cc00640 1 -- 172.21.15.150:0/3605946776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c003520 con 0x7ffb4c0051c0 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.527+0000 7ffb6cc00640 1 -- 172.21.15.150:0/3605946776 reap_dead start 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.528+0000 7ffb6cc00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb4c0051c0 0x7ffb4c00b9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c190 tx=0x7ffb4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a8374736eba42a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.528+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60007c60 con 0x7ffb4c0051c0 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.528+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb60030070 con 0x7ffb4c0051c0 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.528+0000 7ffb66a00640 1 -- 172.21.15.150:0/3605946776 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60005d50 con 0x7ffb4c0051c0 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.528+0000 7ffb6ec00640 1 -- 172.21.15.150:0/3605946776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb4c0051c0 msgr2=0x7ffb4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.528+0000 7ffb6ec00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb4c0051c0 0x7ffb4c00b9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c190 tx=0x7ffb4c005830 comp rx=0 tx=0).stop 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.529+0000 7ffb6ec00640 1 -- 172.21.15.150:0/3605946776 shutdown_connections 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.529+0000 7ffb6ec00640 1 --2- 172.21.15.150:0/3605946776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb4c0051c0 0x7ffb4c00b9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.529+0000 7ffb6ec00640 1 -- 172.21.15.150:0/3605946776 >> 172.21.15.150:0/3605946776 conn(0x7ffb68073480 msgr2=0x7ffb68073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.529+0000 7ffb6ec00640 1 -- 172.21.15.150:0/3605946776 shutdown_connections 2024-04-15T15:24:59.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.529+0000 7ffb6ec00640 1 -- 172.21.15.150:0/3605946776 wait complete. 2024-04-15T15:24:59.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.531+0000 7ffb6ec00640 1 Processor -- start 2024-04-15T15:24:59.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.531+0000 7ffb6ec00640 1 -- start start 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.532+0000 7ffb6ec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c0051c0 0x7ffb6814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.532+0000 7ffb6ec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb680cc670 0x7ffb680ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.532+0000 7ffb6ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6815f740 0x7ffb6815fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.532+0000 7ffb6ec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68163090 con 0x7ffb680cc670 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.532+0000 7ffb6ec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681623a0 con 0x7ffb4c0051c0 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.532+0000 7ffb6ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffb68163210 con 0x7ffb6815f740 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb680cc670 0x7ffb680ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb680cc670 0x7ffb680ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53182/0 (socket says 172.21.15.150:53182) 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 -- 172.21.15.150:0/1020484287 learned_addr learned my addr 172.21.15.150:0/1020484287 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67e00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c0051c0 0x7ffb6814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb6cc00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6815f740 0x7ffb6815fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 -- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6815f740 msgr2=0x7ffb6815fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6815f740 0x7ffb6815fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 -- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c0051c0 msgr2=0x7ffb6814fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c0051c0 0x7ffb6814fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 -- 172.21.15.150:0/1020484287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb680cd110 con 0x7ffb680cc670 2024-04-15T15:24:59.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67e00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c0051c0 0x7ffb6814fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:59.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb6cc00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6815f740 0x7ffb6815fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:24:59.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.533+0000 7ffb67400640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb680cc670 0x7ffb680ccaf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009fb0 tx=0x7ffb580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:59.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.534+0000 7ffb5fe00640 1 -- 172.21.15.150:0/1020484287 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5801a020 con 0x7ffb680cc670 2024-04-15T15:24:59.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.534+0000 7ffb5fe00640 1 -- 172.21.15.150:0/1020484287 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb58016470 con 0x7ffb680cc670 2024-04-15T15:24:59.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.534+0000 7ffb5fe00640 1 -- 172.21.15.150:0/1020484287 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58016860 con 0x7ffb680cc670 2024-04-15T15:24:59.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.534+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb680cd3f0 con 0x7ffb680cc670 2024-04-15T15:24:59.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.534+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb680cd9b0 con 0x7ffb680cc670 2024-04-15T15:24:59.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.535+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb68068c80 con 0x7ffb680cc670 2024-04-15T15:24:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.536+0000 7ffb5fe00640 1 -- 172.21.15.150:0/1020484287 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb58005d10 con 0x7ffb680cc670 2024-04-15T15:24:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.537+0000 7ffb5fe00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ffb4406c920 0x7ffb4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:24:59.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.537+0000 7ffb5fe00640 1 -- 172.21.15.150:0/1020484287 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7ffb5808c610 con 0x7ffb680cc670 2024-04-15T15:24:59.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.537+0000 7ffb67e00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ffb4406c920 0x7ffb4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:24:59.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.538+0000 7ffb67e00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ffb4406c920 0x7ffb4406ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffb50004b60 tx=0x7ffb50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:24:59.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.541+0000 7ffb5fe00640 1 -- 172.21.15.150:0/1020484287 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb58024030 con 0x7ffb680cc670 2024-04-15T15:24:59.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.814+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ffb680c7bb0 con 0x7ffb680cc670 2024-04-15T15:24:59.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.819+0000 7ffb5fe00640 1 -- 172.21.15.150:0/1020484287 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7ffb5805ad80 con 0x7ffb680cc670 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ffb4406c920 msgr2=0x7ffb4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ffb4406c920 0x7ffb4406ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffb50004b60 tx=0x7ffb50009040 comp rx=0 tx=0).stop 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb680cc670 msgr2=0x7ffb680ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb680cc670 0x7ffb680ccaf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009fb0 tx=0x7ffb580115a0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 shutdown_connections 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ffb4406c920 0x7ffb4406ede0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb6815f740 0x7ffb6815fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb680cc670 0x7ffb680ccaf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 --2- 172.21.15.150:0/1020484287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb4c0051c0 0x7ffb6814fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 >> 172.21.15.150:0/1020484287 conn(0x7ffb68073480 msgr2=0x7ffb680d4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:24:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 shutdown_connections 2024-04-15T15:24:59.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:24:59.822+0000 7ffb6ec00640 1 -- 172.21.15.150:0/1020484287 wait complete. 2024-04-15T15:24:59.842 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-15T15:25:00.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.027+0000 7f8eb4a00640 1 Processor -- start 2024-04-15T15:25:00.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.028+0000 7f8eb4a00640 1 -- start start 2024-04-15T15:25:00.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.029+0000 7f8eb4a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb0162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.029+0000 7f8eb4a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8eb0162b70 0x7f8eb00d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.029+0000 7f8eb4a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb00b78f0 con 0x7f8eb00d1a90 2024-04-15T15:25:00.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.029+0000 7f8eb4a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb00d2e50 con 0x7f8eb0162b70 2024-04-15T15:25:00.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.029+0000 7f8eb4a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb0163010 con 0x7f8eb015eab0 2024-04-15T15:25:00.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.030+0000 7f8ea7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8eb0162b70 0x7f8eb00d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.030+0000 7f8ea7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8eb0162b70 0x7f8eb00d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:34770/0 (socket says 172.21.15.150:34770) 2024-04-15T15:25:00.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.030+0000 7f8ea7e00640 1 -- 172.21.15.150:0/591219228 learned_addr learned my addr 172.21.15.150:0/591219228 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:00.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.030+0000 7f8eaf400640 1 --2- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb0162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.030+0000 7f8ea7e00640 1 -- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 msgr2=0x7f8eb0162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:00.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.031+0000 7f8ea7e00640 1 --2- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb0162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.031+0000 7f8ea7e00640 1 -- 172.21.15.150:0/591219228 >> v1:172.21.15.187:6789/0 conn(0x7f8eb00d1a90 legacy=0x7f8eb00b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.031+0000 7f8ea7e00640 1 -- 172.21.15.150:0/591219228 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb01632f0 con 0x7f8eb0162b70 2024-04-15T15:25:00.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.031+0000 7f8eaf400640 1 --2- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb0162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:25:00.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.031+0000 7f8eaea00640 1 -- 172.21.15.150:0/591219228 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340205922 0 0) 0x7f8ea8001490 con 0x7f8eb00d1a90 2024-04-15T15:25:00.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.032+0000 7f8ea7e00640 1 --2- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb0162b70 0x7f8eb00d1550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009870 tx=0x7f8e9802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=405d449fd514bd39 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:00.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.033+0000 7f8eaea00640 1 -- 172.21.15.150:0/591219228 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98034070 con 0x7f8eb0162b70 2024-04-15T15:25:00.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.033+0000 7f8eaea00640 1 -- 172.21.15.150:0/591219228 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e98030840 con 0x7f8eb0162b70 2024-04-15T15:25:00.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.033+0000 7f8eaea00640 1 -- 172.21.15.150:0/591219228 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98030c10 con 0x7f8eb0162b70 2024-04-15T15:25:00.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.034+0000 7f8eb4a00640 1 -- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb0162b70 msgr2=0x7f8eb00d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.034+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb0162b70 0x7f8eb00d1550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009870 tx=0x7f8e9802b980 comp rx=0 tx=0).stop 2024-04-15T15:25:00.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.034+0000 7f8eb4a00640 1 -- 172.21.15.150:0/591219228 shutdown_connections 2024-04-15T15:25:00.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.034+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb0162b70 0x7f8eb00d1550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.034+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/591219228 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb0162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.034+0000 7f8eb4a00640 1 -- 172.21.15.150:0/591219228 >> 172.21.15.150:0/591219228 conn(0x7f8eb0073240 msgr2=0x7f8eb0073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:00.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.034+0000 7f8eb4a00640 1 -- 172.21.15.150:0/591219228 shutdown_connections 2024-04-15T15:25:00.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.034+0000 7f8eb4a00640 1 -- 172.21.15.150:0/591219228 wait complete. 2024-04-15T15:25:00.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.036+0000 7f8eb4a00640 1 Processor -- start 2024-04-15T15:25:00.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.036+0000 7f8eb4a00640 1 -- start start 2024-04-15T15:25:00.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eb4a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb00d1a90 0x7f8eb01614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eb4a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb00ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eb4a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0162b70 0x7f8eb01595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eb4a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb00ba590 con 0x7f8eb0162b70 2024-04-15T15:25:00.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eb4a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb015ee90 con 0x7f8eb00d1a90 2024-04-15T15:25:00.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eb4a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb00ba710 con 0x7f8eb015eab0 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eafe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0162b70 0x7f8eb01595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eafe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0162b70 0x7f8eb01595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53184/0 (socket says 172.21.15.150:53184) 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eafe00640 1 -- 172.21.15.150:0/4137683402 learned_addr learned my addr 172.21.15.150:0/4137683402 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8ea7e00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb00ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eaf400640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb00d1a90 0x7f8eb01614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eafe00640 1 -- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 msgr2=0x7f8eb00ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.037+0000 7f8eafe00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb00ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8eafe00640 1 -- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb00d1a90 msgr2=0x7f8eb01614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8eafe00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb00d1a90 0x7f8eb01614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8eafe00640 1 -- 172.21.15.150:0/4137683402 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb0160400 con 0x7f8eb0162b70 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8ea7e00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb00ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8eaf400640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb00d1a90 0x7f8eb01614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:00.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8eafe00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0162b70 0x7f8eb01595a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c7c0 tx=0x7f8ea800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:00.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8ea7400640 1 -- 172.21.15.150:0/4137683402 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003da0 con 0x7f8eb0162b70 2024-04-15T15:25:00.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8ea7400640 1 -- 172.21.15.150:0/4137683402 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea8005070 con 0x7f8eb0162b70 2024-04-15T15:25:00.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8ea7400640 1 -- 172.21.15.150:0/4137683402 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8005460 con 0x7f8eb0162b70 2024-04-15T15:25:00.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb01606e0 con 0x7f8eb0162b70 2024-04-15T15:25:00.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.038+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb0160ca0 con 0x7f8eb0162b70 2024-04-15T15:25:00.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.040+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb00755c0 con 0x7f8eb0162b70 2024-04-15T15:25:00.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.041+0000 7f8ea7400640 1 -- 172.21.15.150:0/4137683402 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ea8005600 con 0x7f8eb0162b70 2024-04-15T15:25:00.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.042+0000 7f8ea7400640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8e8c06c920 0x7f8e8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.042+0000 7f8ea7400640 1 -- 172.21.15.150:0/4137683402 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f8ea808c1c0 con 0x7f8eb0162b70 2024-04-15T15:25:00.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.042+0000 7f8eaf400640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8e8c06c920 0x7f8e8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.044+0000 7f8eaf400640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8e8c06c920 0x7f8e8c06ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0161ff0 tx=0x7f8ea0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:00.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.045+0000 7f8ea7400640 1 -- 172.21.15.150:0/4137683402 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8eb00b78f0 con 0x7f8eb0162b70 2024-04-15T15:25:00.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.297+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f8eb00b8e60 con 0x7f8eb0162b70 2024-04-15T15:25:00.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.303+0000 7f8ea7400640 1 -- 172.21.15.150:0/4137683402 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f8ea805a9b0 con 0x7f8eb0162b70 2024-04-15T15:25:00.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8e8c06c920 msgr2=0x7f8e8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8e8c06c920 0x7f8e8c06ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0161ff0 tx=0x7f8ea0008040 comp rx=0 tx=0).stop 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0162b70 msgr2=0x7f8eb01595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0162b70 0x7f8eb01595a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c7c0 tx=0x7f8ea800cc80 comp rx=0 tx=0).stop 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 shutdown_connections 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0162b70 0x7f8eb01595a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8e8c06c920 0x7f8e8c06ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb015eab0 0x7f8eb00ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 --2- 172.21.15.150:0/4137683402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb00d1a90 0x7f8eb01614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 >> 172.21.15.150:0/4137683402 conn(0x7f8eb0073240 msgr2=0x7f8eb00c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 shutdown_connections 2024-04-15T15:25:00.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.306+0000 7f8eb4a00640 1 -- 172.21.15.150:0/4137683402 wait complete. 2024-04-15T15:25:00.326 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-15T15:25:00.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.528+0000 7f9a66000640 1 Processor -- start 2024-04-15T15:25:00.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.528+0000 7f9a66000640 1 -- start start 2024-04-15T15:25:00.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.530+0000 7f9a66000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9a600afff0 0x7f9a600cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.530+0000 7f9a66000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a600cfd50 con 0x7f9a60144e40 2024-04-15T15:25:00.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.530+0000 7f9a66000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a600d0030 con 0x7f9a6006f210 2024-04-15T15:25:00.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.530+0000 7f9a66000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9a600b3f30 con 0x7f9a600afff0 2024-04-15T15:25:00.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.530+0000 7f9a56a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9a60144e40 0x7f9a6015bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50908/0 (socket says 172.21.15.150:50908) 2024-04-15T15:25:00.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.530+0000 7f9a56a00640 1 -- 172.21.15.150:0/2700104863 learned_addr learned my addr 172.21.15.150:0/2700104863 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.530+0000 7f9a5f400640 1 --2- 172.21.15.150:0/2700104863 >> v2:172.21.15.187:3300/0 conn(0x7f9a600afff0 0x7f9a600cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 251329497 0 0) 0x7f9a600cfd50 con 0x7f9a60144e40 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a44005730 con 0x7f9a60144e40 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2730918293 0 0) 0x7f9a44005730 con 0x7f9a60144e40 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 >> v2:172.21.15.187:3300/0 conn(0x7f9a600afff0 msgr2=0x7f9a600cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 --2- 172.21.15.150:0/2700104863 >> v2:172.21.15.187:3300/0 conn(0x7f9a600afff0 0x7f9a600cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 >> v1:172.21.15.187:6789/0 conn(0x7f9a6006f210 legacy=0x7f9a600b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a600d0310 con 0x7f9a60144e40 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f9a48001670 con 0x7f9a60144e40 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 >> v1:172.21.15.150:6789/0 conn(0x7f9a60144e40 legacy=0x7f9a6015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a44005bc0 0x7f9a44005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a44006a10 0x7f9a44006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a440073d0 0x7f9a4400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600b3f30 con 0x7f9a440073d0 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4400e370 con 0x7f9a44005bc0 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4400e570 con 0x7f9a44006a10 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.531+0000 7f9a5fe00640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a440073d0 0x7f9a4400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.532+0000 7f9a5f400640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a44005bc0 0x7f9a44005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.532+0000 7f9a5fe00640 1 -- 172.21.15.150:0/2700104863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a44006a10 msgr2=0x7f9a44006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.532+0000 7f9a56a00640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a44006a10 0x7f9a44006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.532+0000 7f9a5fe00640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a44006a10 0x7f9a44006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.532+0000 7f9a5fe00640 1 -- 172.21.15.150:0/2700104863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a44005bc0 msgr2=0x7f9a44005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.532+0000 7f9a5fe00640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a44005bc0 0x7f9a44005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.532+0000 7f9a5fe00640 1 -- 172.21.15.150:0/2700104863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a44005730 con 0x7f9a440073d0 2024-04-15T15:25:00.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.533+0000 7f9a5fe00640 1 -- 172.21.15.150:0/2700104863 reap_dead start 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.533+0000 7f9a5fe00640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a440073d0 0x7f9a4400dbd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a58001e50 tx=0x7f9a44007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eff02e55439012 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.533+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58001a70 con 0x7f9a440073d0 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.533+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a58003bb0 con 0x7f9a440073d0 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.533+0000 7f9a5ea00640 1 -- 172.21.15.150:0/2700104863 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a580050d0 con 0x7f9a440073d0 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.534+0000 7f9a66000640 1 -- 172.21.15.150:0/2700104863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a440073d0 msgr2=0x7f9a4400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.534+0000 7f9a66000640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a440073d0 0x7f9a4400dbd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a58001e50 tx=0x7f9a44007a40 comp rx=0 tx=0).stop 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.534+0000 7f9a66000640 1 -- 172.21.15.150:0/2700104863 shutdown_connections 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.534+0000 7f9a66000640 1 --2- 172.21.15.150:0/2700104863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a440073d0 0x7f9a4400dbd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.534+0000 7f9a66000640 1 -- 172.21.15.150:0/2700104863 >> 172.21.15.150:0/2700104863 conn(0x7f9a600b2460 msgr2=0x7f9a600b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.534+0000 7f9a66000640 1 -- 172.21.15.150:0/2700104863 shutdown_connections 2024-04-15T15:25:00.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.534+0000 7f9a66000640 1 -- 172.21.15.150:0/2700104863 wait complete. 2024-04-15T15:25:00.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.536+0000 7f9a66000640 1 Processor -- start 2024-04-15T15:25:00.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.536+0000 7f9a66000640 1 -- start start 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a66000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a440073d0 0x7f9a600af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a66000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a60141d20 0x7f9a601414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a66000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a600b45a0 0x7f9a600b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a66000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600cf690 con 0x7f9a600b45a0 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a66000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600b4210 con 0x7f9a440073d0 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a66000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9a600cf810 con 0x7f9a60141d20 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a5fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a600b45a0 0x7f9a600b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a5fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a600b45a0 0x7f9a600b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53204/0 (socket says 172.21.15.150:53204) 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a5fe00640 1 -- 172.21.15.150:0/1504688297 learned_addr learned my addr 172.21.15.150:0/1504688297 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a5f400640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a440073d0 0x7f9a600af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.537+0000 7f9a5fe00640 1 -- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a60141d20 msgr2=0x7f9a601414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a5fe00640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a60141d20 0x7f9a601414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a5fe00640 1 -- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a440073d0 msgr2=0x7f9a600af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a5fe00640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a440073d0 0x7f9a600af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a5fe00640 1 -- 172.21.15.150:0/1504688297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a600cb8a0 con 0x7f9a600b45a0 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a5fe00640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a600b45a0 0x7f9a600b4a20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a5802c060 tx=0x7f9a580014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a57e00640 1 -- 172.21.15.150:0/1504688297 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58006d60 con 0x7f9a600b45a0 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a57e00640 1 -- 172.21.15.150:0/1504688297 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a58039480 con 0x7f9a600b45a0 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a57e00640 1 -- 172.21.15.150:0/1504688297 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58039830 con 0x7f9a600b45a0 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a600cbb40 con 0x7f9a600b45a0 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.538+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a600cc100 con 0x7f9a600b45a0 2024-04-15T15:25:00.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.540+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a60068c80 con 0x7f9a600b45a0 2024-04-15T15:25:00.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.541+0000 7f9a57e00640 1 -- 172.21.15.150:0/1504688297 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a580399d0 con 0x7f9a600b45a0 2024-04-15T15:25:00.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.541+0000 7f9a57e00640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9a3c06c8f0 0x7f9a3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:00.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.542+0000 7f9a57e00640 1 -- 172.21.15.150:0/1504688297 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f9a580ad580 con 0x7f9a600b45a0 2024-04-15T15:25:00.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.542+0000 7f9a5f400640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9a3c06c8f0 0x7f9a3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:00.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.543+0000 7f9a5f400640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9a3c06c8f0 0x7f9a3c06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9a500098d0 tx=0x7f9a50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:00.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.547+0000 7f9a57e00640 1 -- 172.21.15.150:0/1504688297 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a5803e070 con 0x7f9a600b45a0 2024-04-15T15:25:00.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.822+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9a600b38d0 con 0x7f9a600b45a0 2024-04-15T15:25:00.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.828+0000 7f9a57e00640 1 -- 172.21.15.150:0/1504688297 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f9a58033030 con 0x7f9a600b45a0 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9a3c06c8f0 msgr2=0x7f9a3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9a3c06c8f0 0x7f9a3c06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9a500098d0 tx=0x7f9a50008040 comp rx=0 tx=0).stop 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a600b45a0 msgr2=0x7f9a600b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a600b45a0 0x7f9a600b4a20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a5802c060 tx=0x7f9a580014a0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 shutdown_connections 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a600b45a0 0x7f9a600b4a20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9a3c06c8f0 0x7f9a3c06edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a60141d20 0x7f9a601414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 --2- 172.21.15.150:0/1504688297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a440073d0 0x7f9a600af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:00.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 >> 172.21.15.150:0/1504688297 conn(0x7f9a600b2460 msgr2=0x7f9a60142840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:00.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 shutdown_connections 2024-04-15T15:25:00.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:00.831+0000 7f9a66000640 1 -- 172.21.15.150:0/1504688297 wait complete. 2024-04-15T15:25:00.850 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-15T15:25:01.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.054+0000 7faca1800640 1 Processor -- start 2024-04-15T15:25:01.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.055+0000 7faca1800640 1 -- start start 2024-04-15T15:25:01.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.056+0000 7faca1800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fac9c0d1550 0x7fac9c0d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.056+0000 7faca1800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c0b78f0 con 0x7fac9c0d32e0 2024-04-15T15:25:01.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.056+0000 7faca1800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c162d40 con 0x7fac9c15eab0 2024-04-15T15:25:01.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.056+0000 7faca1800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c0d1ef0 con 0x7fac9c0d1550 2024-04-15T15:25:01.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.057+0000 7fac9b400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fac9c0d1550 0x7fac9c0d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.057+0000 7fac9b400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fac9c0d1550 0x7fac9c0d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53206/0 (socket says 172.21.15.150:53206) 2024-04-15T15:25:01.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.057+0000 7fac9b400640 1 -- 172.21.15.150:0/3584649646 learned_addr learned my addr 172.21.15.150:0/3584649646 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:01.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.057+0000 7fac9b400640 1 -- 172.21.15.150:0/3584649646 >> v1:172.21.15.187:6789/0 conn(0x7fac9c15eab0 legacy=0x7fac9c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:01.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.057+0000 7fac9b400640 1 -- 172.21.15.150:0/3584649646 >> v1:172.21.15.150:6789/0 conn(0x7fac9c0d32e0 legacy=0x7fac9c0d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.057+0000 7fac9b400640 1 -- 172.21.15.150:0/3584649646 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c163010 con 0x7fac9c0d1550 2024-04-15T15:25:01.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.058+0000 7fac9b400640 1 --2- 172.21.15.150:0/3584649646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d1550 0x7fac9c0d19b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fac84009870 tx=0x7fac8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a92e544b9f686e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:01.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.059+0000 7fac9aa00640 1 -- 172.21.15.150:0/3584649646 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84034070 con 0x7fac9c0d1550 2024-04-15T15:25:01.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.059+0000 7fac9aa00640 1 -- 172.21.15.150:0/3584649646 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac84030860 con 0x7fac9c0d1550 2024-04-15T15:25:01.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.059+0000 7fac9aa00640 1 -- 172.21.15.150:0/3584649646 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84030c30 con 0x7fac9c0d1550 2024-04-15T15:25:01.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.059+0000 7faca1800640 1 -- 172.21.15.150:0/3584649646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d1550 msgr2=0x7fac9c0d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.059+0000 7faca1800640 1 --2- 172.21.15.150:0/3584649646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d1550 0x7fac9c0d19b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fac84009870 tx=0x7fac8402b9a0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.060+0000 7faca1800640 1 -- 172.21.15.150:0/3584649646 shutdown_connections 2024-04-15T15:25:01.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.060+0000 7faca1800640 1 --2- 172.21.15.150:0/3584649646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d1550 0x7fac9c0d19b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.060+0000 7faca1800640 1 -- 172.21.15.150:0/3584649646 >> 172.21.15.150:0/3584649646 conn(0x7fac9c073240 msgr2=0x7fac9c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:01.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.060+0000 7faca1800640 1 -- 172.21.15.150:0/3584649646 shutdown_connections 2024-04-15T15:25:01.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.060+0000 7faca1800640 1 -- 172.21.15.150:0/3584649646 wait complete. 2024-04-15T15:25:01.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.061+0000 7faca1800640 1 Processor -- start 2024-04-15T15:25:01.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.061+0000 7faca1800640 1 -- start start 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.062+0000 7faca1800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac9c0d1550 0x7fac9c1f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.062+0000 7faca1800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d32e0 0x7fac9c144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.062+0000 7faca1800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac9c15eab0 0x7fac9c15f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.062+0000 7faca1800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c0d2ae0 con 0x7fac9c0d32e0 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.062+0000 7faca1800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c0d2e50 con 0x7fac9c15eab0 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.062+0000 7faca1800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c15ee90 con 0x7fac9c0d1550 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d32e0 0x7fac9c144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d32e0 0x7fac9c144940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53220/0 (socket says 172.21.15.150:53220) 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9b400640 1 -- 172.21.15.150:0/2504156169 learned_addr learned my addr 172.21.15.150:0/2504156169 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7faca0e00640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac9c15eab0 0x7fac9c15f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9be00640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac9c0d1550 0x7fac9c1f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9b400640 1 -- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac9c0d1550 msgr2=0x7fac9c1f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9b400640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac9c0d1550 0x7fac9c1f8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9b400640 1 -- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac9c15eab0 msgr2=0x7fac9c15f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9b400640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac9c15eab0 0x7fac9c15f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7fac9b400640 1 -- 172.21.15.150:0/2504156169 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0ca390 con 0x7fac9c0d32e0 2024-04-15T15:25:01.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.063+0000 7faca0e00640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac9c15eab0 0x7fac9c15f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:01.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.064+0000 7fac9b400640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d32e0 0x7fac9c144940 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fac84009870 tx=0x7fac84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:01.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.064+0000 7fac93e00640 1 -- 172.21.15.150:0/2504156169 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84048070 con 0x7fac9c0d32e0 2024-04-15T15:25:01.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.064+0000 7fac93e00640 1 -- 172.21.15.150:0/2504156169 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac84039d90 con 0x7fac9c0d32e0 2024-04-15T15:25:01.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.064+0000 7fac93e00640 1 -- 172.21.15.150:0/2504156169 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84034040 con 0x7fac9c0d32e0 2024-04-15T15:25:01.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.064+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c0ca670 con 0x7fac9c0d32e0 2024-04-15T15:25:01.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.064+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c0cac30 con 0x7fac9c0d32e0 2024-04-15T15:25:01.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.065+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c0755c0 con 0x7fac9c0d32e0 2024-04-15T15:25:01.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.066+0000 7fac93e00640 1 -- 172.21.15.150:0/2504156169 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac84037070 con 0x7fac9c0d32e0 2024-04-15T15:25:01.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.067+0000 7fac93e00640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fac7806c8f0 0x7fac7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.067+0000 7fac93e00640 1 -- 172.21.15.150:0/2504156169 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fac840ad910 con 0x7fac9c0d32e0 2024-04-15T15:25:01.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.068+0000 7fac9be00640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fac7806c8f0 0x7fac7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.069+0000 7fac9be00640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fac7806c8f0 0x7fac7806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac8c007bf0 tx=0x7fac8c007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:01.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.072+0000 7fac93e00640 1 -- 172.21.15.150:0/2504156169 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac8407c080 con 0x7fac9c0d32e0 2024-04-15T15:25:01.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.348+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fac9c15d2a0 con 0x7fac9c0d32e0 2024-04-15T15:25:01.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.353+0000 7fac93e00640 1 -- 172.21.15.150:0/2504156169 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fac8407b980 con 0x7fac9c0d32e0 2024-04-15T15:25:01.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.355+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fac7806c8f0 msgr2=0x7fac7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.357 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.355+0000 7faca1800640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fac7806c8f0 0x7fac7806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac8c007bf0 tx=0x7fac8c007240 comp rx=0 tx=0).stop 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d32e0 msgr2=0x7fac9c144940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d32e0 0x7fac9c144940 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fac84009870 tx=0x7fac84004730 comp rx=0 tx=0).stop 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 shutdown_connections 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac9c15eab0 0x7fac9c15f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac9c0d32e0 0x7fac9c144940 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fac7806c8f0 0x7fac7806edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 --2- 172.21.15.150:0/2504156169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac9c0d1550 0x7fac9c1f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 >> 172.21.15.150:0/2504156169 conn(0x7fac9c073240 msgr2=0x7fac9c0b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 shutdown_connections 2024-04-15T15:25:01.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.356+0000 7faca1800640 1 -- 172.21.15.150:0/2504156169 wait complete. 2024-04-15T15:25:01.378 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-15T15:25:01.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.584+0000 7f34afc00640 1 Processor -- start 2024-04-15T15:25:01.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.584+0000 7f34afc00640 1 -- start start 2024-04-15T15:25:01.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.586+0000 7f34afc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a8162e90 0x7f34a80b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.586+0000 7f34afc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34a8072810 0x7f34a80ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.586+0000 7f34afc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a80b01d0 con 0x7f34a80afd70 2024-04-15T15:25:01.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.586+0000 7f34afc00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f34a80b3900 con 0x7f34a8072810 2024-04-15T15:25:01.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.586+0000 7f34afc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f34a80b3be0 con 0x7f34a8162e90 2024-04-15T15:25:01.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.586+0000 7f34adc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34a8072810 0x7f34a80ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.586+0000 7f34adc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34a8072810 0x7f34a80ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53222/0 (socket says 172.21.15.150:53222) 2024-04-15T15:25:01.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.586+0000 7f34adc00640 1 -- 172.21.15.150:0/515327230 learned_addr learned my addr 172.21.15.150:0/515327230 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:01.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.587+0000 7f34ad200640 1 --2- 172.21.15.150:0/515327230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a8162e90 0x7f34a80b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.587+0000 7f34adc00640 1 -- 172.21.15.150:0/515327230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a8162e90 msgr2=0x7f34a80b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:01.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.587+0000 7f34adc00640 1 --2- 172.21.15.150:0/515327230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a8162e90 0x7f34a80b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.587+0000 7f34adc00640 1 -- 172.21.15.150:0/515327230 >> v1:172.21.15.187:6789/0 conn(0x7f34a80afd70 legacy=0x7f34a8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:01.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.587+0000 7f34adc00640 1 -- 172.21.15.150:0/515327230 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a80b3ec0 con 0x7f34a8072810 2024-04-15T15:25:01.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.588+0000 7f34adc00640 1 --2- 172.21.15.150:0/515327230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8072810 0x7f34a80ce790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009e50 tx=0x7f34a002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7c1d779ef706b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:01.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34a7400640 1 -- 172.21.15.150:0/515327230 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a002c9e0 con 0x7f34a8072810 2024-04-15T15:25:01.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34a7400640 1 -- 172.21.15.150:0/515327230 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a002cb80 con 0x7f34a8072810 2024-04-15T15:25:01.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34a7400640 1 -- 172.21.15.150:0/515327230 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00343f0 con 0x7f34a8072810 2024-04-15T15:25:01.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34afc00640 1 -- 172.21.15.150:0/515327230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8072810 msgr2=0x7f34a80ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34afc00640 1 --2- 172.21.15.150:0/515327230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8072810 0x7f34a80ce790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009e50 tx=0x7f34a002be40 comp rx=0 tx=0).stop 2024-04-15T15:25:01.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34afc00640 1 -- 172.21.15.150:0/515327230 shutdown_connections 2024-04-15T15:25:01.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34afc00640 1 --2- 172.21.15.150:0/515327230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8072810 0x7f34a80ce790 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34afc00640 1 --2- 172.21.15.150:0/515327230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a8162e90 0x7f34a80b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.589+0000 7f34afc00640 1 -- 172.21.15.150:0/515327230 >> 172.21.15.150:0/515327230 conn(0x7f34a80b2460 msgr2=0x7f34a80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:01.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.590+0000 7f34afc00640 1 -- 172.21.15.150:0/515327230 shutdown_connections 2024-04-15T15:25:01.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.590+0000 7f34afc00640 1 -- 172.21.15.150:0/515327230 wait complete. 2024-04-15T15:25:01.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.591+0000 7f34afc00640 1 Processor -- start 2024-04-15T15:25:01.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.591+0000 7f34afc00640 1 -- start start 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.592+0000 7f34afc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8072810 0x7f34a81ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.592+0000 7f34afc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a80afd70 0x7f34a815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.592+0000 7f34afc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8162e90 0x7f34a81eda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.592+0000 7f34afc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34a80cee50 con 0x7f34a8162e90 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.592+0000 7f34afc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f34a80cecd0 con 0x7f34a8072810 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.592+0000 7f34afc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f34a80cefd0 con 0x7f34a80afd70 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.592+0000 7f34adc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8162e90 0x7f34a81eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34adc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8162e90 0x7f34a81eda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53232/0 (socket says 172.21.15.150:53232) 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34adc00640 1 -- 172.21.15.150:0/2991891464 learned_addr learned my addr 172.21.15.150:0/2991891464 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34ad200640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8072810 0x7f34a81ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34a7e00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a80afd70 0x7f34a815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34adc00640 1 -- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a80afd70 msgr2=0x7f34a815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34adc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a80afd70 0x7f34a815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34adc00640 1 -- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8072810 msgr2=0x7f34a81ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34adc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8072810 0x7f34a81ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34adc00640 1 -- 172.21.15.150:0/2991891464 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a81ec7f0 con 0x7f34a8162e90 2024-04-15T15:25:01.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34a7e00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a80afd70 0x7f34a815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:01.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.593+0000 7f34adc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8162e90 0x7f34a81eda70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34a0036040 tx=0x7f34a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:01.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.594+0000 7f34a4c00640 1 -- 172.21.15.150:0/2991891464 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0043020 con 0x7f34a8162e90 2024-04-15T15:25:01.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.594+0000 7f34a4c00640 1 -- 172.21.15.150:0/2991891464 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a003e450 con 0x7f34a8162e90 2024-04-15T15:25:01.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.594+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a81ecad0 con 0x7f34a8162e90 2024-04-15T15:25:01.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.594+0000 7f34a4c00640 1 -- 172.21.15.150:0/2991891464 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0009510 con 0x7f34a8162e90 2024-04-15T15:25:01.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.594+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a81ed090 con 0x7f34a8162e90 2024-04-15T15:25:01.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.595+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a8068c80 con 0x7f34a8162e90 2024-04-15T15:25:01.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.596+0000 7f34a4c00640 1 -- 172.21.15.150:0/2991891464 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34a0046020 con 0x7f34a8162e90 2024-04-15T15:25:01.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.597+0000 7f34a4c00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f348406c8f0 0x7f348406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:01.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.597+0000 7f34a4c00640 1 -- 172.21.15.150:0/2991891464 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f34a00ae060 con 0x7f34a8162e90 2024-04-15T15:25:01.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.597+0000 7f34ad200640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f348406c8f0 0x7f348406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:01.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.598+0000 7f34ad200640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f348406c8f0 0x7f348406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3490006fd0 tx=0x7f3490008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:01.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.601+0000 7f34a4c00640 1 -- 172.21.15.150:0/2991891464 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34a004d030 con 0x7f34a8162e90 2024-04-15T15:25:01.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.877+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f34a8160990 con 0x7f34a8162e90 2024-04-15T15:25:01.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.882+0000 7f34a4c00640 1 -- 172.21.15.150:0/2991891464 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f34a80b01d0 con 0x7f34a8162e90 2024-04-15T15:25:01.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f348406c8f0 msgr2=0x7f348406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f348406c8f0 0x7f348406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3490006fd0 tx=0x7f3490008040 comp rx=0 tx=0).stop 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8162e90 msgr2=0x7f34a81eda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8162e90 0x7f34a81eda70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34a0036040 tx=0x7f34a0004a40 comp rx=0 tx=0).stop 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 shutdown_connections 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a8162e90 0x7f34a81eda70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f348406c8f0 0x7f348406edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34a80afd70 0x7f34a815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 --2- 172.21.15.150:0/2991891464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34a8072810 0x7f34a81ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 >> 172.21.15.150:0/2991891464 conn(0x7f34a80b2460 msgr2=0x7f34a8161490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 shutdown_connections 2024-04-15T15:25:01.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:01.886+0000 7f34afc00640 1 -- 172.21.15.150:0/2991891464 wait complete. 2024-04-15T15:25:01.905 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-15T15:25:02.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.113+0000 7f5ba2200640 1 Processor -- start 2024-04-15T15:25:02.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.114+0000 7f5ba2200640 1 -- start start 2024-04-15T15:25:02.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.115+0000 7f5ba2200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 0x7f5b9c162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.115+0000 7f5ba2200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5b9c162b70 0x7f5b9c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.115+0000 7f5ba2200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b9c0b78f0 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.115+0000 7f5ba2200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f5b9c0d2e50 con 0x7f5b9c162b70 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.115+0000 7f5ba2200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c163010 con 0x7f5b9c15eab0 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.116+0000 7f5b9be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5b9c162b70 0x7f5b9c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.116+0000 7f5b9be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5b9c162b70 0x7f5b9c0d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53236/0 (socket says 172.21.15.150:53236) 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.116+0000 7f5b9be00640 1 -- 172.21.15.150:0/863523675 learned_addr learned my addr 172.21.15.150:0/863523675 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.116+0000 7f5ba0e00640 1 --2- 172.21.15.150:0/863523675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 0x7f5b9c162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.116+0000 7f5b9be00640 1 -- 172.21.15.150:0/863523675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 msgr2=0x7f5b9c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.116+0000 7f5b9be00640 1 --2- 172.21.15.150:0/863523675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 0x7f5b9c162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.116+0000 7f5b9be00640 1 -- 172.21.15.150:0/863523675 >> v1:172.21.15.187:6789/0 conn(0x7f5b9c0d1a90 legacy=0x7f5b9c0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:02.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.116+0000 7f5b9be00640 1 -- 172.21.15.150:0/863523675 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1632f0 con 0x7f5b9c162b70 2024-04-15T15:25:02.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.117+0000 7f5b9be00640 1 --2- 172.21.15.150:0/863523675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c162b70 0x7f5b9c0d1550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009870 tx=0x7f5b8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3202593c285835f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:02.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.117+0000 7f5b9b400640 1 -- 172.21.15.150:0/863523675 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88035020 con 0x7f5b9c162b70 2024-04-15T15:25:02.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.117+0000 7f5b9b400640 1 -- 172.21.15.150:0/863523675 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b88031640 con 0x7f5b9c162b70 2024-04-15T15:25:02.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.117+0000 7f5b9b400640 1 -- 172.21.15.150:0/863523675 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8802d040 con 0x7f5b9c162b70 2024-04-15T15:25:02.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.118+0000 7f5ba2200640 1 -- 172.21.15.150:0/863523675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c162b70 msgr2=0x7f5b9c0d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.118+0000 7f5ba2200640 1 --2- 172.21.15.150:0/863523675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c162b70 0x7f5b9c0d1550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009870 tx=0x7f5b8802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.118+0000 7f5ba2200640 1 -- 172.21.15.150:0/863523675 shutdown_connections 2024-04-15T15:25:02.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.118+0000 7f5ba2200640 1 --2- 172.21.15.150:0/863523675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c162b70 0x7f5b9c0d1550 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.118+0000 7f5ba2200640 1 --2- 172.21.15.150:0/863523675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 0x7f5b9c162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.118+0000 7f5ba2200640 1 -- 172.21.15.150:0/863523675 >> 172.21.15.150:0/863523675 conn(0x7f5b9c073240 msgr2=0x7f5b9c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:02.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.118+0000 7f5ba2200640 1 -- 172.21.15.150:0/863523675 shutdown_connections 2024-04-15T15:25:02.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.118+0000 7f5ba2200640 1 -- 172.21.15.150:0/863523675 wait complete. 2024-04-15T15:25:02.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.120+0000 7f5ba2200640 1 Processor -- start 2024-04-15T15:25:02.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.120+0000 7f5ba2200640 1 -- start start 2024-04-15T15:25:02.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba2200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c0d1a90 0x7f5b9c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba2200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 0x7f5b9c146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba2200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b9c162b70 0x7f5b9c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba2200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c0ba590 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba2200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c15ee90 con 0x7f5b9c162b70 2024-04-15T15:25:02.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba2200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c0ba710 con 0x7f5b9c15eab0 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c0d1a90 0x7f5b9c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c0d1a90 0x7f5b9c0c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53252/0 (socket says 172.21.15.150:53252) 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.121+0000 7f5ba0e00640 1 -- 172.21.15.150:0/295885092 learned_addr learned my addr 172.21.15.150:0/295885092 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba0e00640 1 -- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 msgr2=0x7f5b9c146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba0e00640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 0x7f5b9c146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba1800640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b9c162b70 0x7f5b9c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba0e00640 1 -- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b9c162b70 msgr2=0x7f5b9c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba0e00640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b9c162b70 0x7f5b9c15a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba0e00640 1 -- 172.21.15.150:0/295885092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1e9460 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba0e00640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c0d1a90 0x7f5b9c0c7c30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000d830 tx=0x7f5b9000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:02.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5b98c00640 1 -- 172.21.15.150:0/295885092 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000b6c0 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5b98c00640 1 -- 172.21.15.150:0/295885092 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b90019070 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1e9740 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5b98c00640 1 -- 172.21.15.150:0/295885092 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000ba80 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.122+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c146e40 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.124+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c0755c0 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.124+0000 7f5b98c00640 1 -- 172.21.15.150:0/295885092 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b9001e460 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.125+0000 7f5b98c00640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b7c06c8f0 0x7f5b7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.125+0000 7f5b98c00640 1 -- 172.21.15.150:0/295885092 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f5b9008bf20 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.126+0000 7f5b9be00640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b7c06c8f0 0x7f5b7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.127+0000 7f5b9be00640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b7c06c8f0 0x7f5b7c06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b88036000 tx=0x7f5b88030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:02.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.129+0000 7f5b98c00640 1 -- 172.21.15.150:0/295885092 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b9005a690 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.380+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f5b9c0d2460 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.384+0000 7f5b98c00640 1 -- 172.21.15.150:0/295885092 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f5b90059f90 con 0x7f5b9c0d1a90 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b7c06c8f0 msgr2=0x7f5b7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b7c06c8f0 0x7f5b7c06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b88036000 tx=0x7f5b88030000 comp rx=0 tx=0).stop 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c0d1a90 msgr2=0x7f5b9c0c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c0d1a90 0x7f5b9c0c7c30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000d830 tx=0x7f5b9000dcf0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 shutdown_connections 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b9c162b70 0x7f5b9c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5b7c06c8f0 0x7f5b7c06edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b9c15eab0 0x7f5b9c146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 --2- 172.21.15.150:0/295885092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b9c0d1a90 0x7f5b9c0c7c30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 >> 172.21.15.150:0/295885092 conn(0x7f5b9c073240 msgr2=0x7f5b9c0cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:02.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 shutdown_connections 2024-04-15T15:25:02.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.387+0000 7f5ba2200640 1 -- 172.21.15.150:0/295885092 wait complete. 2024-04-15T15:25:02.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:02.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.612+0000 7f70fec00640 1 Processor -- start 2024-04-15T15:25:02.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.612+0000 7f70fec00640 1 -- start start 2024-04-15T15:25:02.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.613+0000 7f70fec00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70f815d2f0 0x7f70f806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.613+0000 7f70fec00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f815dbb0 con 0x7f70f8072660 2024-04-15T15:25:02.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.614+0000 7f70fec00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f815d8d0 con 0x7f70f80cee00 2024-04-15T15:25:02.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.614+0000 7f70fec00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f70f806f4f0 con 0x7f70f815d2f0 2024-04-15T15:25:02.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.614+0000 7f70fe200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70f815d2f0 0x7f70f806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.614+0000 7f70fe200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70f815d2f0 0x7f70f806e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53254/0 (socket says 172.21.15.150:53254) 2024-04-15T15:25:02.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.614+0000 7f70fe200640 1 -- 172.21.15.150:0/2296080035 learned_addr learned my addr 172.21.15.150:0/2296080035 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:02.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.615+0000 7f70fe200640 1 -- 172.21.15.150:0/2296080035 >> v1:172.21.15.187:6789/0 conn(0x7f70f80cee00 legacy=0x7f70f8071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:02.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.615+0000 7f70f7e00640 1 -- 172.21.15.150:0/2296080035 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504544111 0 0) 0x7f70f815dbb0 con 0x7f70f8072660 2024-04-15T15:25:02.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.615+0000 7f70fe200640 1 -- 172.21.15.150:0/2296080035 >> v1:172.21.15.150:6789/0 conn(0x7f70f8072660 legacy=0x7f70f815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.615+0000 7f70fe200640 1 -- 172.21.15.150:0/2296080035 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f806dde0 con 0x7f70f815d2f0 2024-04-15T15:25:02.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.616+0000 7f70fe200640 1 --2- 172.21.15.150:0/2296080035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f815d2f0 0x7f70f806e8c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009e50 tx=0x7f70f002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b000d0424ad0fc73 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:02.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.616+0000 7f70f7e00640 1 -- 172.21.15.150:0/2296080035 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f002cbb0 con 0x7f70f815d2f0 2024-04-15T15:25:02.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.616+0000 7f70f7e00640 1 -- 172.21.15.150:0/2296080035 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f002cd50 con 0x7f70f815d2f0 2024-04-15T15:25:02.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.616+0000 7f70f7e00640 1 -- 172.21.15.150:0/2296080035 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0034610 con 0x7f70f815d2f0 2024-04-15T15:25:02.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.617+0000 7f70fec00640 1 -- 172.21.15.150:0/2296080035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f815d2f0 msgr2=0x7f70f806e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.617+0000 7f70fec00640 1 --2- 172.21.15.150:0/2296080035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f815d2f0 0x7f70f806e8c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009e50 tx=0x7f70f002c010 comp rx=0 tx=0).stop 2024-04-15T15:25:02.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.617+0000 7f70fec00640 1 -- 172.21.15.150:0/2296080035 shutdown_connections 2024-04-15T15:25:02.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.617+0000 7f70fec00640 1 --2- 172.21.15.150:0/2296080035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f815d2f0 0x7f70f806e8c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.617+0000 7f70fec00640 1 -- 172.21.15.150:0/2296080035 >> 172.21.15.150:0/2296080035 conn(0x7f70f806b930 msgr2=0x7f70f806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:02.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.617+0000 7f70fec00640 1 -- 172.21.15.150:0/2296080035 shutdown_connections 2024-04-15T15:25:02.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.617+0000 7f70fec00640 1 -- 172.21.15.150:0/2296080035 wait complete. 2024-04-15T15:25:02.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.619+0000 7f70fec00640 1 Processor -- start 2024-04-15T15:25:02.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.619+0000 7f70fec00640 1 -- start start 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f8072660 0x7f70f8140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f80cee00 0x7f70f815f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f815d2f0 0x7f70f815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70f815dda0 con 0x7f70f80cee00 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70f806e3a0 con 0x7f70f8072660 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70f815df20 con 0x7f70f815d2f0 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f80cee00 0x7f70f815f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fce00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f80cee00 0x7f70f815f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53260/0 (socket says 172.21.15.150:53260) 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fce00640 1 -- 172.21.15.150:0/687408720 learned_addr learned my addr 172.21.15.150:0/687408720 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fd800640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f8072660 0x7f70f8140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.620+0000 7f70fe200640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f815d2f0 0x7f70f815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fce00640 1 -- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f815d2f0 msgr2=0x7f70f815fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fce00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f815d2f0 0x7f70f815fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fce00640 1 -- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f8072660 msgr2=0x7f70f8140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fce00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f8072660 0x7f70f8140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fce00640 1 -- 172.21.15.150:0/687408720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f814f5a0 con 0x7f70f80cee00 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fe200640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f815d2f0 0x7f70f815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fce00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f80cee00 0x7f70f815f8d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f70e400c460 tx=0x7f70e400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70f5600640 1 -- 172.21.15.150:0/687408720 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4005480 con 0x7f70f80cee00 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70f5600640 1 -- 172.21.15.150:0/687408720 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e40035f0 con 0x7f70f80cee00 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70f5600640 1 -- 172.21.15.150:0/687408720 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4003a00 con 0x7f70f80cee00 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f814f880 con 0x7f70f80cee00 2024-04-15T15:25:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.621+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f814fe40 con 0x7f70f80cee00 2024-04-15T15:25:02.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.623+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f80755c0 con 0x7f70f80cee00 2024-04-15T15:25:02.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.623+0000 7f70f5600640 1 -- 172.21.15.150:0/687408720 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70e4003ba0 con 0x7f70f80cee00 2024-04-15T15:25:02.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.624+0000 7f70f5600640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70d806ca90 0x7f70d806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:02.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.624+0000 7f70f5600640 1 -- 172.21.15.150:0/687408720 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f70e408b570 con 0x7f70f80cee00 2024-04-15T15:25:02.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.625+0000 7f70fd800640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70d806ca90 0x7f70d806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:02.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.626+0000 7f70fd800640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70d806ca90 0x7f70d806ef50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f70ec007fd0 tx=0x7f70ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:02.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.629+0000 7f70f5600640 1 -- 172.21.15.150:0/687408720 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70e4059c50 con 0x7f70f80cee00 2024-04-15T15:25:02.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.830+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70f8156e40 con 0x7f70f80cee00 2024-04-15T15:25:02.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.832+0000 7f70f5600640 1 -- 172.21.15.150:0/687408720 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+12939 (secure 0 0 0) 0x7f70e4059550 con 0x7f70f80cee00 2024-04-15T15:25:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:02.834 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":31,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:24:58.784455+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:02.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70d806ca90 msgr2=0x7f70d806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70d806ca90 0x7f70d806ef50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f70ec007fd0 tx=0x7f70ec009040 comp rx=0 tx=0).stop 2024-04-15T15:25:02.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f80cee00 msgr2=0x7f70f815f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:02.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f80cee00 0x7f70f815f8d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f70e400c460 tx=0x7f70e400c920 comp rx=0 tx=0).stop 2024-04-15T15:25:02.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 shutdown_connections 2024-04-15T15:25:02.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f70d806ca90 0x7f70d806ef50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f815d2f0 0x7f70f815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f80cee00 0x7f70f815f8d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 --2- 172.21.15.150:0/687408720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f8072660 0x7f70f8140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:02.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 >> 172.21.15.150:0/687408720 conn(0x7f70f806b930 msgr2=0x7f70f806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:02.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.834+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 shutdown_connections 2024-04-15T15:25:02.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:02.835+0000 7f70fec00640 1 -- 172.21.15.150:0/687408720 wait complete. 2024-04-15T15:25:02.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:25:02.858 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:25:03.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.066+0000 7f7bce600640 1 Processor -- start 2024-04-15T15:25:03.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.066+0000 7f7bce600640 1 -- start start 2024-04-15T15:25:03.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.067+0000 7f7bce600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7bc8160530 0x7f7bc8166760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bce600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7bc8160910 0x7f7bc8166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bce600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc806f590 con 0x7f7bc806e910 2024-04-15T15:25:03.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bce600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc80ce4f0 con 0x7f7bc8160530 2024-04-15T15:25:03.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bce600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc80ce7d0 con 0x7f7bc8160910 2024-04-15T15:25:03.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bcd200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7bc8160530 0x7f7bc8166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bcdc00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7bc806e910 0x7f7bc806eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50924/0 (socket says 172.21.15.150:50924) 2024-04-15T15:25:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bcdc00640 1 -- 172.21.15.150:0/2624314963 learned_addr learned my addr 172.21.15.150:0/2624314963 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bc7e00640 1 --2- 172.21.15.150:0/2624314963 >> v2:172.21.15.187:3300/0 conn(0x7f7bc8160910 0x7f7bc8166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bcd200640 1 -- 172.21.15.150:0/2624314963 >> v2:172.21.15.187:3300/0 conn(0x7f7bc8160910 msgr2=0x7f7bc8166ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bcd200640 1 --2- 172.21.15.150:0/2624314963 >> v2:172.21.15.187:3300/0 conn(0x7f7bc8160910 0x7f7bc8166ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.068+0000 7f7bcd200640 1 -- 172.21.15.150:0/2624314963 >> v1:172.21.15.150:6789/0 conn(0x7f7bc806e910 legacy=0x7f7bc806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.069+0000 7f7bcd200640 1 -- 172.21.15.150:0/2624314963 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc806f870 con 0x7f7bc8160530 2024-04-15T15:25:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.069+0000 7f7bc7e00640 1 --2- 172.21.15.150:0/2624314963 >> v2:172.21.15.187:3300/0 conn(0x7f7bc8160910 0x7f7bc8166ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.069+0000 7f7bc7400640 1 -- 172.21.15.150:0/2624314963 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701034839 0 0) 0x7f7bc806f590 con 0x7f7bc806e910 2024-04-15T15:25:03.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bcd200640 1 --2- 172.21.15.150:0/2624314963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8160530 0x7f7bc8166760 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80b68c0 tx=0x7f7bbc02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc8242ef26401c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bc7400640 1 -- 172.21.15.150:0/2624314963 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc038070 con 0x7f7bc8160530 2024-04-15T15:25:03.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bc7400640 1 -- 172.21.15.150:0/2624314963 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bbc031710 con 0x7f7bc8160530 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bc7400640 1 -- 172.21.15.150:0/2624314963 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc031ae0 con 0x7f7bc8160530 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bce600640 1 -- 172.21.15.150:0/2624314963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8160530 msgr2=0x7f7bc8166760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bce600640 1 --2- 172.21.15.150:0/2624314963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8160530 0x7f7bc8166760 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80b68c0 tx=0x7f7bbc02b7b0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bce600640 1 -- 172.21.15.150:0/2624314963 shutdown_connections 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bce600640 1 --2- 172.21.15.150:0/2624314963 >> v2:172.21.15.187:3300/0 conn(0x7f7bc8160910 0x7f7bc8166ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bce600640 1 --2- 172.21.15.150:0/2624314963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8160530 0x7f7bc8166760 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.070+0000 7f7bce600640 1 -- 172.21.15.150:0/2624314963 >> 172.21.15.150:0/2624314963 conn(0x7f7bc80b23a0 msgr2=0x7f7bc80b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.071+0000 7f7bce600640 1 -- 172.21.15.150:0/2624314963 shutdown_connections 2024-04-15T15:25:03.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.071+0000 7f7bce600640 1 -- 172.21.15.150:0/2624314963 wait complete. 2024-04-15T15:25:03.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.072+0000 7f7bce600640 1 Processor -- start 2024-04-15T15:25:03.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.072+0000 7f7bce600640 1 -- start start 2024-04-15T15:25:03.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.073+0000 7f7bce600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc806e910 0x7f7bc80af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.073+0000 7f7bce600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc8160530 0x7f7bc815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.073+0000 7f7bce600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815b5d0 0x7f7bc80a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.073+0000 7f7bce600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80cf360 con 0x7f7bc806e910 2024-04-15T15:25:03.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.073+0000 7f7bce600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80cf1e0 con 0x7f7bc815b5d0 2024-04-15T15:25:03.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.073+0000 7f7bce600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc80cf4e0 con 0x7f7bc8160530 2024-04-15T15:25:03.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc806e910 0x7f7bc80af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc806e910 0x7f7bc80af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53278/0 (socket says 172.21.15.150:53278) 2024-04-15T15:25:03.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 -- 172.21.15.150:0/1819354943 learned_addr learned my addr 172.21.15.150:0/1819354943 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:03.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 -- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc8160530 msgr2=0x7f7bc815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc8160530 0x7f7bc815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 -- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815b5d0 msgr2=0x7f7bc80a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815b5d0 0x7f7bc80a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 -- 172.21.15.150:0/1819354943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc815d0a0 con 0x7f7bc806e910 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.074+0000 7f7bcd200640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc806e910 0x7f7bc80af220 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc036040 tx=0x7f7bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.075+0000 7f7bc4c00640 1 -- 172.21.15.150:0/1819354943 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc038040 con 0x7f7bc806e910 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.075+0000 7f7bc4c00640 1 -- 172.21.15.150:0/1819354943 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bbc033a40 con 0x7f7bc806e910 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.075+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc815d320 con 0x7f7bc806e910 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.075+0000 7f7bc4c00640 1 -- 172.21.15.150:0/1819354943 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc033df0 con 0x7f7bc806e910 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.075+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc815d8e0 con 0x7f7bc806e910 2024-04-15T15:25:03.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.076+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc815db20 con 0x7f7bc806e910 2024-04-15T15:25:03.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.077+0000 7f7bc4c00640 1 -- 172.21.15.150:0/1819354943 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bbc03b070 con 0x7f7bc806e910 2024-04-15T15:25:03.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.078+0000 7f7bc4c00640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ba806c8f0 0x7f7ba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.078+0000 7f7bc4c00640 1 -- 172.21.15.150:0/1819354943 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f7bbc0ad490 con 0x7f7bc806e910 2024-04-15T15:25:03.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.079+0000 7f7bc7e00640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ba806c8f0 0x7f7ba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.080+0000 7f7bc7e00640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ba806c8f0 0x7f7ba806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4006fd0 tx=0x7f7bb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.083+0000 7f7bc4c00640 1 -- 172.21.15.150:0/1819354943 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bc815db20 con 0x7f7bc806e910 2024-04-15T15:25:03.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.280+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7bc80755c0 con 0x7f7bc806e910 2024-04-15T15:25:03.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.281+0000 7f7bc4c00640 1 -- 172.21.15.150:0/1819354943 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) ==== 93+0+10 (secure 0 0 0) 0x7f7bc815d0a0 con 0x7f7bc806e910 2024-04-15T15:25:03.283 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:25:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.283+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ba806c8f0 msgr2=0x7f7ba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.283+0000 7f7bce600640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ba806c8f0 0x7f7ba806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4006fd0 tx=0x7f7bb4008040 comp rx=0 tx=0).stop 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.283+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc806e910 msgr2=0x7f7bc80af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.283+0000 7f7bce600640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc806e910 0x7f7bc80af220 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc036040 tx=0x7f7bbc0046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.284+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 shutdown_connections 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.284+0000 7f7bce600640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bc815b5d0 0x7f7bc80a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.284+0000 7f7bce600640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ba806c8f0 0x7f7ba806edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.284+0000 7f7bce600640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7bc8160530 0x7f7bc815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.284+0000 7f7bce600640 1 --2- 172.21.15.150:0/1819354943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc806e910 0x7f7bc80af220 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.284+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 >> 172.21.15.150:0/1819354943 conn(0x7f7bc80b23a0 msgr2=0x7f7bc8165580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.284+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 shutdown_connections 2024-04-15T15:25:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.284+0000 7f7bce600640 1 -- 172.21.15.150:0/1819354943 wait complete. 2024-04-15T15:25:03.305 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-15T15:25:03.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.514+0000 7f3ff8600640 1 Processor -- start 2024-04-15T15:25:03.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.514+0000 7f3ff8600640 1 -- start start 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.515+0000 7f3ff8600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3ff01481f0 0x7f3ff015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.515+0000 7f3ff8600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ff00affc0 0x7f3ff00721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00b5560 0x7f3ff0072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff8600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff015ffc0 con 0x7f3ff00affc0 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff8600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff01485d0 con 0x7f3ff01481f0 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff00aef20 con 0x7f3ff00b5560 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ff00affc0 0x7f3ff00721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ff00affc0 0x7f3ff00721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53292/0 (socket says 172.21.15.150:53292) 2024-04-15T15:25:03.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5200640 1 -- 172.21.15.150:0/4244859190 learned_addr learned my addr 172.21.15.150:0/4244859190 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5200640 1 -- 172.21.15.150:0/4244859190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00b5560 msgr2=0x7f3ff0072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5c00640 1 --2- 172.21.15.150:0/4244859190 >> v2:172.21.15.187:3300/0 conn(0x7f3ff01481f0 0x7f3ff015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff6600640 1 --2- 172.21.15.150:0/4244859190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00b5560 0x7f3ff0072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5200640 1 --2- 172.21.15.150:0/4244859190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00b5560 0x7f3ff0072720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5200640 1 -- 172.21.15.150:0/4244859190 >> v2:172.21.15.187:3300/0 conn(0x7f3ff01481f0 msgr2=0x7f3ff015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5200640 1 --2- 172.21.15.150:0/4244859190 >> v2:172.21.15.187:3300/0 conn(0x7f3ff01481f0 0x7f3ff015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5200640 1 -- 172.21.15.150:0/4244859190 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff00af200 con 0x7f3ff00affc0 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.516+0000 7f3ff5c00640 1 --2- 172.21.15.150:0/4244859190 >> v2:172.21.15.187:3300/0 conn(0x7f3ff01481f0 0x7f3ff015fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:03.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.517+0000 7f3ff5200640 1 --2- 172.21.15.150:0/4244859190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00affc0 0x7f3ff00721e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0076c20 tx=0x7f3fdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2d88615df5ba60 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.518+0000 7f3fefe00640 1 -- 172.21.15.150:0/4244859190 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc035020 con 0x7f3ff00affc0 2024-04-15T15:25:03.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.518+0000 7f3fefe00640 1 -- 172.21.15.150:0/4244859190 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc031870 con 0x7f3ff00affc0 2024-04-15T15:25:03.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.518+0000 7f3fefe00640 1 -- 172.21.15.150:0/4244859190 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc031c40 con 0x7f3ff00affc0 2024-04-15T15:25:03.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.518+0000 7f3ff8600640 1 -- 172.21.15.150:0/4244859190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00affc0 msgr2=0x7f3ff00721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.518+0000 7f3ff8600640 1 --2- 172.21.15.150:0/4244859190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00affc0 0x7f3ff00721e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0076c20 tx=0x7f3fdc02b910 comp rx=0 tx=0).stop 2024-04-15T15:25:03.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.519+0000 7f3ff8600640 1 -- 172.21.15.150:0/4244859190 shutdown_connections 2024-04-15T15:25:03.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.519+0000 7f3ff8600640 1 --2- 172.21.15.150:0/4244859190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00b5560 0x7f3ff0072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.519+0000 7f3ff8600640 1 --2- 172.21.15.150:0/4244859190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00affc0 0x7f3ff00721e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.519+0000 7f3ff8600640 1 --2- 172.21.15.150:0/4244859190 >> v2:172.21.15.187:3300/0 conn(0x7f3ff01481f0 0x7f3ff015fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.519+0000 7f3ff8600640 1 -- 172.21.15.150:0/4244859190 >> 172.21.15.150:0/4244859190 conn(0x7f3ff00b27b0 msgr2=0x7f3ff00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:03.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.519+0000 7f3ff8600640 1 -- 172.21.15.150:0/4244859190 shutdown_connections 2024-04-15T15:25:03.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.519+0000 7f3ff8600640 1 -- 172.21.15.150:0/4244859190 wait complete. 2024-04-15T15:25:03.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.520+0000 7f3ff8600640 1 Processor -- start 2024-04-15T15:25:03.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.520+0000 7f3ff8600640 1 -- start start 2024-04-15T15:25:03.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.521+0000 7f3ff8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00affc0 0x7f3ff0152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.521+0000 7f3ff8600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00b5560 0x7f3ff013bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.521+0000 7f3ff8600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ff01481f0 0x7f3ff006f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.521+0000 7f3ff8600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff00afaa0 con 0x7f3ff00b5560 2024-04-15T15:25:03.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.521+0000 7f3ff8600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff00af7c0 con 0x7f3ff01481f0 2024-04-15T15:25:03.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.521+0000 7f3ff8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff00b38d0 con 0x7f3ff00affc0 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00b5560 0x7f3ff013bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00b5560 0x7f3ff013bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53304/0 (socket says 172.21.15.150:53304) 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 -- 172.21.15.150:0/3432576386 learned_addr learned my addr 172.21.15.150:0/3432576386 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5c00640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00affc0 0x7f3ff0152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 -- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00affc0 msgr2=0x7f3ff0152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00affc0 0x7f3ff0152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 -- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ff01481f0 msgr2=0x7f3ff006f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ff01481f0 0x7f3ff006f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 -- 172.21.15.150:0/3432576386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff0141150 con 0x7f3ff00b5560 2024-04-15T15:25:03.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.522+0000 7f3ff5200640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00b5560 0x7f3ff013bf70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc02b950 tx=0x7f3fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.523+0000 7f3fed600640 1 -- 172.21.15.150:0/3432576386 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc040070 con 0x7f3ff00b5560 2024-04-15T15:25:03.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.523+0000 7f3fed600640 1 -- 172.21.15.150:0/3432576386 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc03a420 con 0x7f3ff00b5560 2024-04-15T15:25:03.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.523+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff0141430 con 0x7f3ff00b5560 2024-04-15T15:25:03.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.523+0000 7f3fed600640 1 -- 172.21.15.150:0/3432576386 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc035040 con 0x7f3ff00b5560 2024-04-15T15:25:03.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.523+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff0152ec0 con 0x7f3ff00b5560 2024-04-15T15:25:03.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.524+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff0068c80 con 0x7f3ff00b5560 2024-04-15T15:25:03.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.525+0000 7f3fed600640 1 -- 172.21.15.150:0/3432576386 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fdc04b060 con 0x7f3ff00b5560 2024-04-15T15:25:03.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.526+0000 7f3fed600640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3fd006c920 0x7f3fd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.526+0000 7f3fed600640 1 -- 172.21.15.150:0/3432576386 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f3fdc0ad750 con 0x7f3ff00b5560 2024-04-15T15:25:03.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.527+0000 7f3ff5c00640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3fd006c920 0x7f3fd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.528+0000 7f3ff5c00640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3fd006c920 0x7f3fd006ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400b840 tx=0x7f3fe4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.530+0000 7f3fed600640 1 -- 172.21.15.150:0/3432576386 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fdc07bed0 con 0x7f3ff00b5560 2024-04-15T15:25:03.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.721+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3ff00cf130 con 0x7f3ff00b5560 2024-04-15T15:25:03.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.722+0000 7f3fed600640 1 -- 172.21.15.150:0/3432576386 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7f3fdc07b7d0 con 0x7f3ff00b5560 2024-04-15T15:25:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:03.724 INFO:teuthology.orchestra.run.smithi150.stdout:[ 2024-04-15T15:25:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "poolnum": 1, 2024-04-15T15:25:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "poolname": ".mgr" 2024-04-15T15:25:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:25:03.724 INFO:teuthology.orchestra.run.smithi150.stdout:] 2024-04-15T15:25:03.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3fd006c920 msgr2=0x7f3fd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3fd006c920 0x7f3fd006ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400b840 tx=0x7f3fe4008040 comp rx=0 tx=0).stop 2024-04-15T15:25:03.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00b5560 msgr2=0x7f3ff013bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00b5560 0x7f3ff013bf70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc02b950 tx=0x7f3fdc0046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 shutdown_connections 2024-04-15T15:25:03.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ff01481f0 0x7f3ff006f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ff00b5560 0x7f3ff013bf70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3fd006c920 0x7f3fd006ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 --2- 172.21.15.150:0/3432576386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ff00affc0 0x7f3ff0152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 >> 172.21.15.150:0/3432576386 conn(0x7f3ff00b27b0 msgr2=0x7f3ff00b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:03.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 shutdown_connections 2024-04-15T15:25:03.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.725+0000 7f3ff8600640 1 -- 172.21.15.150:0/3432576386 wait complete. 2024-04-15T15:25:03.744 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-15T15:25:03.744 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-15T15:25:03.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.958+0000 7f75b0e00640 1 Processor -- start 2024-04-15T15:25:03.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.958+0000 7f75b0e00640 1 -- start start 2024-04-15T15:25:03.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75b0e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f75ac15ea90 0x7f75ac0d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75b0e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 0x7f75ac0d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75b0e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75ac0cf170 0x7f75ac0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75b0e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f75ac0cfe30 con 0x7f75ac15ea90 2024-04-15T15:25:03.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75b0e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f75ac0cfb60 con 0x7f75ac0cf170 2024-04-15T15:25:03.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75b0e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75ac0d2a40 con 0x7f75ac0d3800 2024-04-15T15:25:03.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75a9e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f75ac15ea90 0x7f75ac0d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75a9e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f75ac15ea90 0x7f75ac0d3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53316/0 (socket says 172.21.15.150:53316) 2024-04-15T15:25:03.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.960+0000 7f75a9e00640 1 -- 172.21.15.150:0/218487775 learned_addr learned my addr 172.21.15.150:0/218487775 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:03.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.961+0000 7f75aa800640 1 --2- 172.21.15.150:0/218487775 >> v2:172.21.15.187:3300/0 conn(0x7f75ac0cf170 0x7f75ac0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.961+0000 7f75a9400640 1 --2- 172.21.15.150:0/218487775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 0x7f75ac0d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.961+0000 7f75a9e00640 1 -- 172.21.15.150:0/218487775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 msgr2=0x7f75ac0d4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:03.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.961+0000 7f75a9e00640 1 --2- 172.21.15.150:0/218487775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 0x7f75ac0d4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.961+0000 7f75a9e00640 1 -- 172.21.15.150:0/218487775 >> v2:172.21.15.187:3300/0 conn(0x7f75ac0cf170 msgr2=0x7f75ac0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:03.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.961+0000 7f75a9e00640 1 --2- 172.21.15.150:0/218487775 >> v2:172.21.15.187:3300/0 conn(0x7f75ac0cf170 0x7f75ac0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.961+0000 7f75a9e00640 1 -- 172.21.15.150:0/218487775 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac0d2d20 con 0x7f75ac15ea90 2024-04-15T15:25:03.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.961+0000 7f75aa800640 1 --2- 172.21.15.150:0/218487775 >> v2:172.21.15.187:3300/0 conn(0x7f75ac0cf170 0x7f75ac0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:03.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.962+0000 7f75a9e00640 1 --2- 172.21.15.150:0/218487775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac15ea90 0x7f75ac0d3e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f759c009870 tx=0x7f759c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece61a81ba17b311 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.962+0000 7f75a8a00640 1 -- 172.21.15.150:0/218487775 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c035020 con 0x7f75ac15ea90 2024-04-15T15:25:03.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.962+0000 7f75a8a00640 1 -- 172.21.15.150:0/218487775 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f759c031640 con 0x7f75ac15ea90 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.962+0000 7f75a8a00640 1 -- 172.21.15.150:0/218487775 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c02d040 con 0x7f75ac15ea90 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 -- 172.21.15.150:0/218487775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac15ea90 msgr2=0x7f75ac0d3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 --2- 172.21.15.150:0/218487775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac15ea90 0x7f75ac0d3e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f759c009870 tx=0x7f759c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 -- 172.21.15.150:0/218487775 shutdown_connections 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 --2- 172.21.15.150:0/218487775 >> v2:172.21.15.187:3300/0 conn(0x7f75ac0cf170 0x7f75ac0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 --2- 172.21.15.150:0/218487775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 0x7f75ac0d4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 --2- 172.21.15.150:0/218487775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac15ea90 0x7f75ac0d3e50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 -- 172.21.15.150:0/218487775 >> 172.21.15.150:0/218487775 conn(0x7f75ac073480 msgr2=0x7f75ac073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 -- 172.21.15.150:0/218487775 shutdown_connections 2024-04-15T15:25:03.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.963+0000 7f75b0e00640 1 -- 172.21.15.150:0/218487775 wait complete. 2024-04-15T15:25:03.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.965+0000 7f75b0e00640 1 Processor -- start 2024-04-15T15:25:03.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.966+0000 7f75b0e00640 1 -- start start 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75b0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac0cf170 0x7f75ac145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75b0e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 0x7f75ac0d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75b0e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac15ea90 0x7f75ac0d4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75b0e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac15ee70 con 0x7f75ac0cf170 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75b0e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac0d32e0 con 0x7f75ac15ea90 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75b0e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75ac0b6420 con 0x7f75ac0d3800 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75a9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac0cf170 0x7f75ac145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75a9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac0cf170 0x7f75ac145f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53322/0 (socket says 172.21.15.150:53322) 2024-04-15T15:25:03.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75a9e00640 1 -- 172.21.15.150:0/2390880058 learned_addr learned my addr 172.21.15.150:0/2390880058 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:03.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75a9e00640 1 -- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 msgr2=0x7f75ac0d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75a9e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 0x7f75ac0d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75a9e00640 1 -- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac15ea90 msgr2=0x7f75ac0d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75a9e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac15ea90 0x7f75ac0d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.967+0000 7f75a9e00640 1 -- 172.21.15.150:0/2390880058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac1577d0 con 0x7f75ac0cf170 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.968+0000 7f75a9e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac0cf170 0x7f75ac145f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f759c000c00 tx=0x7f759c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.968+0000 7f75a2000640 1 -- 172.21.15.150:0/2390880058 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c040070 con 0x7f75ac0cf170 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.968+0000 7f75a2000640 1 -- 172.21.15.150:0/2390880058 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f759c0399f0 con 0x7f75ac0cf170 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.968+0000 7f75a2000640 1 -- 172.21.15.150:0/2390880058 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c035040 con 0x7f75ac0cf170 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.968+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac143540 con 0x7f75ac0cf170 2024-04-15T15:25:03.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.968+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac143b00 con 0x7f75ac0cf170 2024-04-15T15:25:03.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.969+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ac0d1360 con 0x7f75ac0cf170 2024-04-15T15:25:03.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.971+0000 7f75a2000640 1 -- 172.21.15.150:0/2390880058 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f759c04b060 con 0x7f75ac0cf170 2024-04-15T15:25:03.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.972+0000 7f75a2000640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f758806c920 0x7f758806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:03.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.972+0000 7f75a2000640 1 -- 172.21.15.150:0/2390880058 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f759c0ad030 con 0x7f75ac0cf170 2024-04-15T15:25:03.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.972+0000 7f75a9400640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f758806c920 0x7f758806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:03.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.973+0000 7f75a9400640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f758806c920 0x7f758806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f75ac146760 tx=0x7f7594008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:03.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:03.976+0000 7f75a2000640 1 -- 172.21.15.150:0/2390880058 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75ac0d1360 con 0x7f75ac0cf170 2024-04-15T15:25:04.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.176+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f75ac068db0 con 0x7f75ac0cf170 2024-04-15T15:25:04.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.853+0000 7f75a2000640 1 -- 172.21.15.150:0/2390880058 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7f759c07b840 con 0x7f75ac0cf170 2024-04-15T15:25:04.855 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_metadata' created 2024-04-15T15:25:04.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.856+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f758806c920 msgr2=0x7f758806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:04.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.856+0000 7f75b0e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f758806c920 0x7f758806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f75ac146760 tx=0x7f7594008040 comp rx=0 tx=0).stop 2024-04-15T15:25:04.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.856+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac0cf170 msgr2=0x7f75ac145f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:04.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.856+0000 7f75b0e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac0cf170 0x7f75ac145f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f759c000c00 tx=0x7f759c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:04.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.857+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 shutdown_connections 2024-04-15T15:25:04.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.857+0000 7f75b0e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac15ea90 0x7f75ac0d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:04.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.857+0000 7f75b0e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f758806c920 0x7f758806ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:04.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.857+0000 7f75b0e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75ac0d3800 0x7f75ac0d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:04.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.857+0000 7f75b0e00640 1 --2- 172.21.15.150:0/2390880058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75ac0cf170 0x7f75ac145f30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:04.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.857+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 >> 172.21.15.150:0/2390880058 conn(0x7f75ac073480 msgr2=0x7f75ac0b9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:04.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.857+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 shutdown_connections 2024-04-15T15:25:04.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:04.857+0000 7f75b0e00640 1 -- 172.21.15.150:0/2390880058 wait complete. 2024-04-15T15:25:04.886 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-15T15:25:05.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.086+0000 7fc48c800640 1 Processor -- start 2024-04-15T15:25:05.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.086+0000 7fc48c800640 1 -- start start 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.088+0000 7fc48c800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc48406fc80 0x7fc48415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.088+0000 7fc48c800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc484072150 0x7fc484072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.088+0000 7fc48c800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc48415d520 con 0x7fc48415a440 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.088+0000 7fc48c800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fc48406e280 con 0x7fc48406fc80 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.088+0000 7fc48c800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc484072ad0 con 0x7fc484072150 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.088+0000 7fc489400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc48406fc80 0x7fc48415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.089+0000 7fc489e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc48415a440 0x7fc4840cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50930/0 (socket says 172.21.15.150:50930) 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.089+0000 7fc489400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc48406fc80 0x7fc48415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53326/0 (socket says 172.21.15.150:53326) 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.089+0000 7fc489e00640 1 -- 172.21.15.150:0/782802969 learned_addr learned my addr 172.21.15.150:0/782802969 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.089+0000 7fc48a800640 1 --2- 172.21.15.150:0/782802969 >> v2:172.21.15.187:3300/0 conn(0x7fc484072150 0x7fc484072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:05.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.089+0000 7fc489400640 1 -- 172.21.15.150:0/782802969 >> v2:172.21.15.187:3300/0 conn(0x7fc484072150 msgr2=0x7fc484072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:05.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.089+0000 7fc489400640 1 --2- 172.21.15.150:0/782802969 >> v2:172.21.15.187:3300/0 conn(0x7fc484072150 0x7fc484072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.089+0000 7fc489400640 1 -- 172.21.15.150:0/782802969 >> v1:172.21.15.150:6789/0 conn(0x7fc48415a440 legacy=0x7fc4840cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:05.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.089+0000 7fc489400640 1 -- 172.21.15.150:0/782802969 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48415d800 con 0x7fc48406fc80 2024-04-15T15:25:05.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.090+0000 7fc489400640 1 --2- 172.21.15.150:0/782802969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48406fc80 0x7fc48415cef0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc48406fba0 tx=0x7fc47402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263bbf62203522d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:05.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.091+0000 7fc488a00640 1 -- 172.21.15.150:0/782802969 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474035020 con 0x7fc48406fc80 2024-04-15T15:25:05.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.091+0000 7fc488a00640 1 -- 172.21.15.150:0/782802969 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc474031830 con 0x7fc48406fc80 2024-04-15T15:25:05.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.091+0000 7fc488a00640 1 -- 172.21.15.150:0/782802969 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474031c00 con 0x7fc48406fc80 2024-04-15T15:25:05.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.091+0000 7fc48c800640 1 -- 172.21.15.150:0/782802969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48406fc80 msgr2=0x7fc48415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:05.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.091+0000 7fc48c800640 1 --2- 172.21.15.150:0/782802969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48406fc80 0x7fc48415cef0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc48406fba0 tx=0x7fc47402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.091+0000 7fc48c800640 1 -- 172.21.15.150:0/782802969 shutdown_connections 2024-04-15T15:25:05.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.091+0000 7fc48c800640 1 --2- 172.21.15.150:0/782802969 >> v2:172.21.15.187:3300/0 conn(0x7fc484072150 0x7fc484072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.092+0000 7fc48c800640 1 --2- 172.21.15.150:0/782802969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48406fc80 0x7fc48415cef0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.092+0000 7fc48c800640 1 -- 172.21.15.150:0/782802969 >> 172.21.15.150:0/782802969 conn(0x7fc4840b2460 msgr2=0x7fc4840b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:05.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.092+0000 7fc48c800640 1 -- 172.21.15.150:0/782802969 shutdown_connections 2024-04-15T15:25:05.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.092+0000 7fc48c800640 1 -- 172.21.15.150:0/782802969 wait complete. 2024-04-15T15:25:05.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.093+0000 7fc48c800640 1 Processor -- start 2024-04-15T15:25:05.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.094+0000 7fc48c800640 1 -- start start 2024-04-15T15:25:05.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.094+0000 7fc48c800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc48406fc80 0x7fc48415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:05.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.094+0000 7fc48c800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc484072150 0x7fc48415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:05.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48c800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48415a440 0x7fc48415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:05.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48c800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc48415a820 con 0x7fc48415a440 2024-04-15T15:25:05.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48c800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4840cd7f0 con 0x7fc48406fc80 2024-04-15T15:25:05.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48c800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc48415ddc0 con 0x7fc484072150 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48415a440 0x7fc48415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48415a440 0x7fc48415aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53336/0 (socket says 172.21.15.150:53336) 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48a800640 1 -- 172.21.15.150:0/2223870755 learned_addr learned my addr 172.21.15.150:0/2223870755 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc489400640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc484072150 0x7fc48415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc489e00640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc48406fc80 0x7fc48415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48a800640 1 -- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc484072150 msgr2=0x7fc48415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48a800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc484072150 0x7fc48415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48a800640 1 -- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc48406fc80 msgr2=0x7fc48415c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48a800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc48406fc80 0x7fc48415c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc48a800640 1 -- 172.21.15.150:0/2223870755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc484132190 con 0x7fc48415a440 2024-04-15T15:25:05.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc489400640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc484072150 0x7fc48415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:05.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.095+0000 7fc489e00640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc48406fc80 0x7fc48415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:05.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.096+0000 7fc48a800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48415a440 0x7fc48415aa30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc480007e70 tx=0x7fc48000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:05.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.096+0000 7fc47e000640 1 -- 172.21.15.150:0/2223870755 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480014620 con 0x7fc48415a440 2024-04-15T15:25:05.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.096+0000 7fc47e000640 1 -- 172.21.15.150:0/2223870755 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc480019070 con 0x7fc48415a440 2024-04-15T15:25:05.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.096+0000 7fc47e000640 1 -- 172.21.15.150:0/2223870755 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480014a20 con 0x7fc48415a440 2024-04-15T15:25:05.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.096+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4841e5220 con 0x7fc48415a440 2024-04-15T15:25:05.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.096+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4841e57e0 con 0x7fc48415a440 2024-04-15T15:25:05.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.097+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc484132360 con 0x7fc48415a440 2024-04-15T15:25:05.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.099+0000 7fc47e000640 1 -- 172.21.15.150:0/2223870755 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc480014bc0 con 0x7fc48415a440 2024-04-15T15:25:05.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.099+0000 7fc47e000640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:05.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.100+0000 7fc47e000640 1 -- 172.21.15.150:0/2223870755 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7fc48008c2d0 con 0x7fc48415a440 2024-04-15T15:25:05.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.100+0000 7fc489e00640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:05.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.102+0000 7fc489e00640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc46806c8f0 0x7fc46806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc478000f60 tx=0x7fc478009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:05.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.105+0000 7fc47e000640 1 -- 172.21.15.150:0/2223870755 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc480011260 con 0x7fc48415a440 2024-04-15T15:25:05.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.309+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc484068c80 con 0x7fc48415a440 2024-04-15T15:25:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.859+0000 7fc47e000640 1 -- 172.21.15.150:0/2223870755 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7fc48005a8d0 con 0x7fc48415a440 2024-04-15T15:25:05.862 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data' created 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc46806c8f0 msgr2=0x7fc46806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc46806c8f0 0x7fc46806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc478000f60 tx=0x7fc478009040 comp rx=0 tx=0).stop 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48415a440 msgr2=0x7fc48415aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48415a440 0x7fc48415aa30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc480007e70 tx=0x7fc48000e4b0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 shutdown_connections 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48415a440 0x7fc48415aa30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc484072150 0x7fc48415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 --2- 172.21.15.150:0/2223870755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc48406fc80 0x7fc48415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:05.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.863+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 >> 172.21.15.150:0/2223870755 conn(0x7fc4840b2460 msgr2=0x7fc4840ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:05.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.864+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 shutdown_connections 2024-04-15T15:25:05.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:05.864+0000 7fc48c800640 1 -- 172.21.15.150:0/2223870755 wait complete. 2024-04-15T15:25:05.902 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-15T15:25:06.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.084+0000 7fc93f400640 1 Processor -- start 2024-04-15T15:25:06.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.084+0000 7fc93f400640 1 -- start start 2024-04-15T15:25:06.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.086+0000 7fc93f400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc93815a440 0x7fc93815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:06.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.086+0000 7fc93f400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93815d520 con 0x7fc93806fc80 2024-04-15T15:25:06.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.086+0000 7fc93f400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93815d800 con 0x7fc938072150 2024-04-15T15:25:06.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.086+0000 7fc93f400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fc93806e280 con 0x7fc93815a440 2024-04-15T15:25:06.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.086+0000 7fc937e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc93806fc80 0x7fc9380cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50940/0 (socket says 172.21.15.150:50940) 2024-04-15T15:25:06.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.086+0000 7fc937e00640 1 -- 172.21.15.150:0/3694718663 learned_addr learned my addr 172.21.15.150:0/3694718663 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:06.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.086+0000 7fc93ca00640 1 --2- 172.21.15.150:0/3694718663 >> v2:172.21.15.150:3300/0 conn(0x7fc93815a440 0x7fc93815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:06.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.087+0000 7fc937400640 1 -- 172.21.15.150:0/3694718663 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3217085450 0 0) 0x7fc93815d520 con 0x7fc93806fc80 2024-04-15T15:25:06.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.087+0000 7fc937400640 1 -- 172.21.15.150:0/3694718663 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc920003520 con 0x7fc93806fc80 2024-04-15T15:25:06.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.087+0000 7fc93ca00640 1 -- 172.21.15.150:0/3694718663 >> v1:172.21.15.187:6789/0 conn(0x7fc938072150 legacy=0x7fc93806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:06.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.087+0000 7fc93ca00640 1 -- 172.21.15.150:0/3694718663 >> v1:172.21.15.150:6789/0 conn(0x7fc93806fc80 legacy=0x7fc9380cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:06.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.087+0000 7fc93ca00640 1 -- 172.21.15.150:0/3694718663 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93815dae0 con 0x7fc93815a440 2024-04-15T15:25:06.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.089+0000 7fc93ca00640 1 --2- 172.21.15.150:0/3694718663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93815a440 0x7fc93815cef0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc93806e400 tx=0x7fc92c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28381a0b558bd4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:06.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.089+0000 7fc937400640 1 -- 172.21.15.150:0/3694718663 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c034070 con 0x7fc93815a440 2024-04-15T15:25:06.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.089+0000 7fc937400640 1 -- 172.21.15.150:0/3694718663 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c030610 con 0x7fc93815a440 2024-04-15T15:25:06.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.089+0000 7fc937400640 1 -- 172.21.15.150:0/3694718663 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0309c0 con 0x7fc93815a440 2024-04-15T15:25:06.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.090+0000 7fc93f400640 1 -- 172.21.15.150:0/3694718663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93815a440 msgr2=0x7fc93815cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:06.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.090+0000 7fc93f400640 1 --2- 172.21.15.150:0/3694718663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93815a440 0x7fc93815cef0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc93806e400 tx=0x7fc92c02b960 comp rx=0 tx=0).stop 2024-04-15T15:25:06.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.090+0000 7fc93f400640 1 -- 172.21.15.150:0/3694718663 shutdown_connections 2024-04-15T15:25:06.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.090+0000 7fc93f400640 1 --2- 172.21.15.150:0/3694718663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93815a440 0x7fc93815cef0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:06.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.090+0000 7fc93f400640 1 -- 172.21.15.150:0/3694718663 >> 172.21.15.150:0/3694718663 conn(0x7fc9380b2460 msgr2=0x7fc9380b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:06.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.090+0000 7fc93f400640 1 -- 172.21.15.150:0/3694718663 shutdown_connections 2024-04-15T15:25:06.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.090+0000 7fc93f400640 1 -- 172.21.15.150:0/3694718663 wait complete. 2024-04-15T15:25:06.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.092+0000 7fc93f400640 1 Processor -- start 2024-04-15T15:25:06.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.092+0000 7fc93f400640 1 -- start start 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93806fc80 0x7fc9381f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93f400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc938072150 0x7fc938145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93f400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc93815a440 0x7fc93815ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93f400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc93815ddc0 con 0x7fc93806fc80 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93f400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc93815a820 con 0x7fc93815a440 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93f400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc93806f510 con 0x7fc938072150 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93806fc80 0x7fc9381f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93806fc80 0x7fc9381f7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53356/0 (socket says 172.21.15.150:53356) 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.093+0000 7fc93ca00640 1 -- 172.21.15.150:0/997417791 learned_addr learned my addr 172.21.15.150:0/997417791 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.094+0000 7fc937e00640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc938072150 0x7fc938145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.094+0000 7fc93d400640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc93815a440 0x7fc93815ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.094+0000 7fc93ca00640 1 -- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc938072150 msgr2=0x7fc938145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:06.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.094+0000 7fc93ca00640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc938072150 0x7fc938145050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:06.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.094+0000 7fc93ca00640 1 -- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc93815a440 msgr2=0x7fc93815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:06.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.094+0000 7fc93ca00640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc93815a440 0x7fc93815ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:06.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.094+0000 7fc93ca00640 1 -- 172.21.15.150:0/997417791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9380c60c0 con 0x7fc93806fc80 2024-04-15T15:25:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.095+0000 7fc93ca00640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93806fc80 0x7fc9381f7d50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009890 tx=0x7fc92c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.095+0000 7fc934c00640 1 -- 172.21.15.150:0/997417791 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c047070 con 0x7fc93806fc80 2024-04-15T15:25:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.095+0000 7fc934c00640 1 -- 172.21.15.150:0/997417791 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c030410 con 0x7fc93806fc80 2024-04-15T15:25:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.095+0000 7fc934c00640 1 -- 172.21.15.150:0/997417791 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c034040 con 0x7fc93806fc80 2024-04-15T15:25:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.095+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9381f85a0 con 0x7fc93806fc80 2024-04-15T15:25:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.095+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9381f8b60 con 0x7fc93806fc80 2024-04-15T15:25:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.096+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9380c64c0 con 0x7fc93806fc80 2024-04-15T15:25:06.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.097+0000 7fc934c00640 1 -- 172.21.15.150:0/997417791 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc92c04a070 con 0x7fc93806fc80 2024-04-15T15:25:06.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.098+0000 7fc934c00640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc91806c8f0 0x7fc91806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:06.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.098+0000 7fc934c00640 1 -- 172.21.15.150:0/997417791 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7fc92c0ad490 con 0x7fc93806fc80 2024-04-15T15:25:06.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.099+0000 7fc937e00640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc91806c8f0 0x7fc91806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.100+0000 7fc937e00640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc91806c8f0 0x7fc91806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc938159a90 tx=0x7fc924009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:06.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.103+0000 7fc934c00640 1 -- 172.21.15.150:0/997417791 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc92c07b350 con 0x7fc93806fc80 2024-04-15T15:25:06.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.341+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc9380b5830 con 0x7fc93806fc80 2024-04-15T15:25:06.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.918+0000 7fc934c00640 1 -- 172.21.15.150:0/997417791 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fc92c07f650 con 0x7fc93806fc80 2024-04-15T15:25:06.923 INFO:teuthology.orchestra.run.smithi150.stderr:new fs with metadata pool 4 and data pool 5 2024-04-15T15:25:06.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.926+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc91806c8f0 msgr2=0x7fc91806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:06.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.926+0000 7fc93f400640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc91806c8f0 0x7fc91806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc938159a90 tx=0x7fc924009040 comp rx=0 tx=0).stop 2024-04-15T15:25:06.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.926+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93806fc80 msgr2=0x7fc9381f7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:06.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.926+0000 7fc93f400640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93806fc80 0x7fc9381f7d50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009890 tx=0x7fc92c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:06.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.927+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 shutdown_connections 2024-04-15T15:25:06.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.927+0000 7fc93f400640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc93815a440 0x7fc93815ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:06.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.927+0000 7fc93f400640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc91806c8f0 0x7fc91806edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:06.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.927+0000 7fc93f400640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc938072150 0x7fc938145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:06.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.927+0000 7fc93f400640 1 --2- 172.21.15.150:0/997417791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc93806fc80 0x7fc9381f7d50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:06.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.927+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 >> 172.21.15.150:0/997417791 conn(0x7fc9380b2460 msgr2=0x7fc93806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:06.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.928+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 shutdown_connections 2024-04-15T15:25:06.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:06.929+0000 7fc93f400640 1 -- 172.21.15.150:0/997417791 wait complete. 2024-04-15T15:25:06.964 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:07.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.142+0000 7f7267a00640 1 Processor -- start 2024-04-15T15:25:07.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.142+0000 7f7267a00640 1 -- start start 2024-04-15T15:25:07.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.143+0000 7f7267a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7260070210 0x7f72600b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.143+0000 7f7267a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72600b5020 0x7f726015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7267a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f726006f940 con 0x7f726015b6d0 2024-04-15T15:25:07.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7267a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f726006e8a0 con 0x7f72600b5020 2024-04-15T15:25:07.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7267a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f726006eb80 con 0x7f7260070210 2024-04-15T15:25:07.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7265a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72600b5020 0x7f726015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7265a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72600b5020 0x7f726015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55122/0 (socket says 172.21.15.150:55122) 2024-04-15T15:25:07.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7265a00640 1 -- 172.21.15.150:0/2656982639 learned_addr learned my addr 172.21.15.150:0/2656982639 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:07.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f725fe00640 1 --2- 172.21.15.150:0/2656982639 >> v2:172.21.15.187:3300/0 conn(0x7f7260070210 0x7f72600b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7265a00640 1 -- 172.21.15.150:0/2656982639 >> v2:172.21.15.187:3300/0 conn(0x7f7260070210 msgr2=0x7f72600b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:07.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7265a00640 1 --2- 172.21.15.150:0/2656982639 >> v2:172.21.15.187:3300/0 conn(0x7f7260070210 0x7f72600b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7265a00640 1 -- 172.21.15.150:0/2656982639 >> v1:172.21.15.187:6789/0 conn(0x7f726015b6d0 legacy=0x7f72600cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:07.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.144+0000 7f7265a00640 1 -- 172.21.15.150:0/2656982639 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726006ee60 con 0x7f72600b5020 2024-04-15T15:25:07.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.145+0000 7f7265a00640 1 --2- 172.21.15.150:0/2656982639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72600b5020 0x7f726015bab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f72600cf470 tx=0x7f725802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3c0a06187c96f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:07.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.146+0000 7f725f400640 1 -- 172.21.15.150:0/2656982639 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725802e070 con 0x7f72600b5020 2024-04-15T15:25:07.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.146+0000 7f725f400640 1 -- 172.21.15.150:0/2656982639 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7258037070 con 0x7f72600b5020 2024-04-15T15:25:07.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.146+0000 7f725f400640 1 -- 172.21.15.150:0/2656982639 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258031a00 con 0x7f72600b5020 2024-04-15T15:25:07.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.146+0000 7f7267a00640 1 -- 172.21.15.150:0/2656982639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72600b5020 msgr2=0x7f726015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.146+0000 7f7267a00640 1 --2- 172.21.15.150:0/2656982639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72600b5020 0x7f726015bab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f72600cf470 tx=0x7f725802b880 comp rx=0 tx=0).stop 2024-04-15T15:25:07.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.147+0000 7f7267a00640 1 -- 172.21.15.150:0/2656982639 shutdown_connections 2024-04-15T15:25:07.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.147+0000 7f7267a00640 1 --2- 172.21.15.150:0/2656982639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72600b5020 0x7f726015bab0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.147+0000 7f7267a00640 1 --2- 172.21.15.150:0/2656982639 >> v2:172.21.15.187:3300/0 conn(0x7f7260070210 0x7f72600b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.147+0000 7f7267a00640 1 -- 172.21.15.150:0/2656982639 >> 172.21.15.150:0/2656982639 conn(0x7f72600b27b0 msgr2=0x7f72600b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:07.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.147+0000 7f7267a00640 1 -- 172.21.15.150:0/2656982639 shutdown_connections 2024-04-15T15:25:07.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.147+0000 7f7267a00640 1 -- 172.21.15.150:0/2656982639 wait complete. 2024-04-15T15:25:07.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.148+0000 7f7267a00640 1 Processor -- start 2024-04-15T15:25:07.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.149+0000 7f7267a00640 1 -- start start 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.149+0000 7f7267a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260070210 0x7f72600a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.149+0000 7f7267a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72600b5020 0x7f7260147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.149+0000 7f7267a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f726015b6d0 0x7f72600cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.149+0000 7f7267a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f72600ce420 con 0x7f7260070210 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.149+0000 7f7267a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f726006f420 con 0x7f72600b5020 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.149+0000 7f7267a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f726015bff0 con 0x7f726015b6d0 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260070210 0x7f72600a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260070210 0x7f72600a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55124/0 (socket says 172.21.15.150:55124) 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 -- 172.21.15.150:0/4097074764 learned_addr learned my addr 172.21.15.150:0/4097074764 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 -- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f726015b6d0 msgr2=0x7f72600cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f726015b6d0 0x7f72600cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 -- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72600b5020 msgr2=0x7f7260147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72600b5020 0x7f7260147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 -- 172.21.15.150:0/4097074764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7260148850 con 0x7f7260070210 2024-04-15T15:25:07.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.150+0000 7f7265000640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260070210 0x7f72600a9c60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f724800b420 tx=0x7f724800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:07.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.151+0000 7f725cc00640 1 -- 172.21.15.150:0/4097074764 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248011830 con 0x7f7260070210 2024-04-15T15:25:07.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.151+0000 7f725cc00640 1 -- 172.21.15.150:0/4097074764 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7248016070 con 0x7f7260070210 2024-04-15T15:25:07.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.151+0000 7f725cc00640 1 -- 172.21.15.150:0/4097074764 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248011c10 con 0x7f7260070210 2024-04-15T15:25:07.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.151+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7260148b30 con 0x7f7260070210 2024-04-15T15:25:07.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.151+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7260145f80 con 0x7f7260070210 2024-04-15T15:25:07.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.152+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7260068c80 con 0x7f7260070210 2024-04-15T15:25:07.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.153+0000 7f725cc00640 1 -- 172.21.15.150:0/4097074764 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f724801b460 con 0x7f7260070210 2024-04-15T15:25:07.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.154+0000 7f725cc00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f723c06c8f0 0x7f723c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.155+0000 7f725cc00640 1 -- 172.21.15.150:0/4097074764 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f724808b680 con 0x7f7260070210 2024-04-15T15:25:07.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.155+0000 7f725fe00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f723c06c8f0 0x7f723c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.156+0000 7f725fe00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f723c06c8f0 0x7f723c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f72500070b0 tx=0x7f7250007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:07.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.159+0000 7f725cc00640 1 -- 172.21.15.150:0/4097074764 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7248020020 con 0x7f7260070210 2024-04-15T15:25:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.366+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72600d0560 con 0x7f7260070210 2024-04-15T15:25:07.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.370+0000 7f725cc00640 1 -- 172.21.15.150:0/4097074764 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16730 (secure 0 0 0) 0x7f7248059ab0 con 0x7f7260070210 2024-04-15T15:25:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:07.372 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":34,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:06.343688+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:07.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f723c06c8f0 msgr2=0x7f723c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f723c06c8f0 0x7f723c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f72500070b0 tx=0x7f7250007040 comp rx=0 tx=0).stop 2024-04-15T15:25:07.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260070210 msgr2=0x7f72600a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260070210 0x7f72600a9c60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f724800b420 tx=0x7f724800b8e0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 shutdown_connections 2024-04-15T15:25:07.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f723c06c8f0 0x7f723c06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f726015b6d0 0x7f72600cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72600b5020 0x7f7260147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 --2- 172.21.15.150:0/4097074764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260070210 0x7f72600a9c60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 >> 172.21.15.150:0/4097074764 conn(0x7f72600b27b0 msgr2=0x7f72600b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:07.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 shutdown_connections 2024-04-15T15:25:07.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.373+0000 7f7267a00640 1 -- 172.21.15.150:0/4097074764 wait complete. 2024-04-15T15:25:07.394 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.602+0000 7f0ed8200640 1 Processor -- start 2024-04-15T15:25:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.602+0000 7f0ed8200640 1 -- start start 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed8200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ed01461f0 0x7f0ed00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed8200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed00b4be0 con 0x7f0ed00cedf0 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed8200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed00a9640 con 0x7f0ed01461f0 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed00cf250 con 0x7f0ed00b3fe0 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed5800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ed01461f0 0x7f0ed00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed5800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ed01461f0 0x7f0ed00ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55130/0 (socket says 172.21.15.150:55130) 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed5800640 1 -- 172.21.15.150:0/1404234738 learned_addr learned my addr 172.21.15.150:0/1404234738 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.604+0000 7f0ed6200640 1 --2- 172.21.15.150:0/1404234738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.605+0000 7f0ed5800640 1 -- 172.21.15.150:0/1404234738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 msgr2=0x7f0ed015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.605+0000 7f0ed5800640 1 --2- 172.21.15.150:0/1404234738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed015d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.605+0000 7f0ed5800640 1 -- 172.21.15.150:0/1404234738 >> v1:172.21.15.187:6789/0 conn(0x7f0ed00cedf0 legacy=0x7f0ed00b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.605+0000 7f0ed5800640 1 -- 172.21.15.150:0/1404234738 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed00b4620 con 0x7f0ed01461f0 2024-04-15T15:25:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.605+0000 7f0ed6200640 1 --2- 172.21.15.150:0/1404234738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:25:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.606+0000 7f0ed5800640 1 --2- 172.21.15.150:0/1404234738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed00ce8b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009870 tx=0x7f0ec402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e12b9d5a6834ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:07.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.607+0000 7f0ecfe00640 1 -- 172.21.15.150:0/1404234738 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec402f840 con 0x7f0ed01461f0 2024-04-15T15:25:07.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.607+0000 7f0ecfe00640 1 -- 172.21.15.150:0/1404234738 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec402f9e0 con 0x7f0ed01461f0 2024-04-15T15:25:07.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.607+0000 7f0ecfe00640 1 -- 172.21.15.150:0/1404234738 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec402fdb0 con 0x7f0ed01461f0 2024-04-15T15:25:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.607+0000 7f0ed8200640 1 -- 172.21.15.150:0/1404234738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 msgr2=0x7f0ed00ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.607+0000 7f0ed8200640 1 --2- 172.21.15.150:0/1404234738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed00ce8b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009870 tx=0x7f0ec402ba20 comp rx=0 tx=0).stop 2024-04-15T15:25:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.608+0000 7f0ed8200640 1 -- 172.21.15.150:0/1404234738 shutdown_connections 2024-04-15T15:25:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.608+0000 7f0ed8200640 1 --2- 172.21.15.150:0/1404234738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.608+0000 7f0ed8200640 1 --2- 172.21.15.150:0/1404234738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed00ce8b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.608+0000 7f0ed8200640 1 -- 172.21.15.150:0/1404234738 >> 172.21.15.150:0/1404234738 conn(0x7f0ed00b27b0 msgr2=0x7f0ed00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.608+0000 7f0ed8200640 1 -- 172.21.15.150:0/1404234738 shutdown_connections 2024-04-15T15:25:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.608+0000 7f0ed8200640 1 -- 172.21.15.150:0/1404234738 wait complete. 2024-04-15T15:25:07.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.609+0000 7f0ed8200640 1 Processor -- start 2024-04-15T15:25:07.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.610+0000 7f0ed8200640 1 -- start start 2024-04-15T15:25:07.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.610+0000 7f0ed8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed01f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.610+0000 7f0ed8200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed00cedf0 0x7f0ed00a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.610+0000 7f0ed8200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed01e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.610+0000 7f0ed8200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed015d290 con 0x7f0ed01461f0 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.610+0000 7f0ed8200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed015cfb0 con 0x7f0ed00cedf0 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.610+0000 7f0ed8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed01465d0 con 0x7f0ed00b3fe0 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed6200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed01e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed6200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed01e7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55140/0 (socket says 172.21.15.150:55140) 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed6200640 1 -- 172.21.15.150:0/4007035832 learned_addr learned my addr 172.21.15.150:0/4007035832 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed5800640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed01f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed4e00640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed00cedf0 0x7f0ed00a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed6200640 1 -- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 msgr2=0x7f0ed01f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed6200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed01f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed6200640 1 -- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed00cedf0 msgr2=0x7f0ed00a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed6200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed00cedf0 0x7f0ed00a97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.611+0000 7f0ed6200640 1 -- 172.21.15.150:0/4007035832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed00ccef0 con 0x7f0ed01461f0 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.612+0000 7f0ed6200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed01e7ec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8007ef0 tx=0x7f0ec800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.612+0000 7f0ecd600640 1 -- 172.21.15.150:0/4007035832 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec801a070 con 0x7f0ed01461f0 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.612+0000 7f0ecd600640 1 -- 172.21.15.150:0/4007035832 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec8003b90 con 0x7f0ed01461f0 2024-04-15T15:25:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.612+0000 7f0ecd600640 1 -- 172.21.15.150:0/4007035832 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8004030 con 0x7f0ed01461f0 2024-04-15T15:25:07.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.612+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed00cd1d0 con 0x7f0ed01461f0 2024-04-15T15:25:07.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.612+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed00cd790 con 0x7f0ed01461f0 2024-04-15T15:25:07.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.613+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed0068c80 con 0x7f0ed01461f0 2024-04-15T15:25:07.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.614+0000 7f0ecd600640 1 -- 172.21.15.150:0/4007035832 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ec801d070 con 0x7f0ed01461f0 2024-04-15T15:25:07.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.615+0000 7f0ecd600640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0eb006c8f0 0x7f0eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:07.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.615+0000 7f0ecd600640 1 -- 172.21.15.150:0/4007035832 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0ec808bdb0 con 0x7f0ed01461f0 2024-04-15T15:25:07.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.616+0000 7f0ed5800640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0eb006c8f0 0x7f0eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:07.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.617+0000 7f0ed5800640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0eb006c8f0 0x7f0eb006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4036000 tx=0x7f0ec4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:07.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.627+0000 7f0ecd600640 1 -- 172.21.15.150:0/4007035832 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ec8090070 con 0x7f0ed01461f0 2024-04-15T15:25:07.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.828+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ed015fa10 con 0x7f0ed01461f0 2024-04-15T15:25:07.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.830+0000 7f0ecd600640 1 -- 172.21.15.150:0/4007035832 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16730 (secure 0 0 0) 0x7f0ec8010070 con 0x7f0ed01461f0 2024-04-15T15:25:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:07.832 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":34,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:06.343688+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:07.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.832+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0eb006c8f0 msgr2=0x7f0eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.832+0000 7f0ed8200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0eb006c8f0 0x7f0eb006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4036000 tx=0x7f0ec4008030 comp rx=0 tx=0).stop 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.832+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 msgr2=0x7f0ed01e7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.832+0000 7f0ed8200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed01e7ec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8007ef0 tx=0x7f0ec800d4a0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.833+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 shutdown_connections 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.833+0000 7f0ed8200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed01461f0 0x7f0ed01e7ec0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.833+0000 7f0ed8200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed00cedf0 0x7f0ed00a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.833+0000 7f0ed8200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0eb006c8f0 0x7f0eb006edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.833+0000 7f0ed8200640 1 --2- 172.21.15.150:0/4007035832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ed00b3fe0 0x7f0ed01f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.833+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 >> 172.21.15.150:0/4007035832 conn(0x7f0ed00b27b0 msgr2=0x7f0ed00c60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.833+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 shutdown_connections 2024-04-15T15:25:07.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:07.833+0000 7f0ed8200640 1 -- 172.21.15.150:0/4007035832 wait complete. 2024-04-15T15:25:07.853 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-15T15:25:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.066+0000 7f5ed8c00640 1 Processor -- start 2024-04-15T15:25:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.066+0000 7f5ed8c00640 1 -- start start 2024-04-15T15:25:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.068+0000 7f5ed8c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ed415a440 0x7f5ed415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.068+0000 7f5ed8c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5ed406fc80 0x7f5ed40cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.068+0000 7f5ed8c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed415d520 con 0x7f5ed40cd620 2024-04-15T15:25:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.068+0000 7f5ed8c00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed406e280 con 0x7f5ed415a440 2024-04-15T15:25:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.068+0000 7f5ed8c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed415d800 con 0x7f5ed406fc80 2024-04-15T15:25:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.068+0000 7f5ed1e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ed415a440 0x7f5ed415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.069+0000 7f5ed1e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ed415a440 0x7f5ed415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55142/0 (socket says 172.21.15.150:55142) 2024-04-15T15:25:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.069+0000 7f5ed1e00640 1 -- 172.21.15.150:0/4188749649 learned_addr learned my addr 172.21.15.150:0/4188749649 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.069+0000 7f5ed1e00640 1 -- 172.21.15.150:0/4188749649 >> v2:172.21.15.187:3300/0 conn(0x7f5ed406fc80 msgr2=0x7f5ed40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.069+0000 7f5ec9400640 1 --2- 172.21.15.150:0/4188749649 >> v2:172.21.15.187:3300/0 conn(0x7f5ed406fc80 0x7f5ed40cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.069+0000 7f5ed1e00640 1 --2- 172.21.15.150:0/4188749649 >> v2:172.21.15.187:3300/0 conn(0x7f5ed406fc80 0x7f5ed40cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.069+0000 7f5ed1e00640 1 -- 172.21.15.150:0/4188749649 >> v1:172.21.15.187:6789/0 conn(0x7f5ed40cd620 legacy=0x7f5ed4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.069+0000 7f5ed1e00640 1 -- 172.21.15.150:0/4188749649 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed415dae0 con 0x7f5ed415a440 2024-04-15T15:25:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.069+0000 7f5ec9400640 1 --2- 172.21.15.150:0/4188749649 >> v2:172.21.15.187:3300/0 conn(0x7f5ed406fc80 0x7f5ed40cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.070+0000 7f5ed1e00640 1 --2- 172.21.15.150:0/4188749649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed415a440 0x7f5ed415cef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ccb70 tx=0x7f5ec402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ceae7025046548 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:08.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed1400640 1 -- 172.21.15.150:0/4188749649 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec402e070 con 0x7f5ed415a440 2024-04-15T15:25:08.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed1400640 1 -- 172.21.15.150:0/4188749649 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec4037070 con 0x7f5ed415a440 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed1400640 1 -- 172.21.15.150:0/4188749649 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4031a00 con 0x7f5ed415a440 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed8c00640 1 -- 172.21.15.150:0/4188749649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed415a440 msgr2=0x7f5ed415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/4188749649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed415a440 0x7f5ed415cef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ccb70 tx=0x7f5ec402b880 comp rx=0 tx=0).stop 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed8c00640 1 -- 172.21.15.150:0/4188749649 shutdown_connections 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/4188749649 >> v2:172.21.15.187:3300/0 conn(0x7f5ed406fc80 0x7f5ed40cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/4188749649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed415a440 0x7f5ed415cef0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed8c00640 1 -- 172.21.15.150:0/4188749649 >> 172.21.15.150:0/4188749649 conn(0x7f5ed40b2460 msgr2=0x7f5ed40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.071+0000 7f5ed8c00640 1 -- 172.21.15.150:0/4188749649 shutdown_connections 2024-04-15T15:25:08.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.072+0000 7f5ed8c00640 1 -- 172.21.15.150:0/4188749649 wait complete. 2024-04-15T15:25:08.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.073+0000 7f5ed8c00640 1 Processor -- start 2024-04-15T15:25:08.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.073+0000 7f5ed8c00640 1 -- start start 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.074+0000 7f5ed8c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed406fc80 0x7f5ed4142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.074+0000 7f5ed8c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ed40cd620 0x7f5ed40cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.074+0000 7f5ed8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ed415a440 0x7f5ed415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.074+0000 7f5ed8c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed406f510 con 0x7f5ed406fc80 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.074+0000 7f5ed8c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed415ddc0 con 0x7f5ed40cd620 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.074+0000 7f5ed8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed4072860 con 0x7f5ed415a440 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.075+0000 7f5ed1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed406fc80 0x7f5ed4142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.075+0000 7f5ed1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed406fc80 0x7f5ed4142140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55158/0 (socket says 172.21.15.150:55158) 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.075+0000 7f5ed1e00640 1 -- 172.21.15.150:0/1847182489 learned_addr learned my addr 172.21.15.150:0/1847182489 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:08.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.075+0000 7f5ed1e00640 1 -- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ed415a440 msgr2=0x7f5ed415ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:08.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.075+0000 7f5ed1e00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ed415a440 0x7f5ed415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.075+0000 7f5ed1e00640 1 -- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ed40cd620 msgr2=0x7f5ed40cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:08.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.075+0000 7f5ed1e00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ed40cd620 0x7f5ed40cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.075+0000 7f5ed1e00640 1 -- 172.21.15.150:0/1847182489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41f7bd0 con 0x7f5ed406fc80 2024-04-15T15:25:08.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.076+0000 7f5ed1e00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed406fc80 0x7f5ed4142140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5ec402b8c0 tx=0x7f5ec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:08.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.076+0000 7f5ecaa00640 1 -- 172.21.15.150:0/1847182489 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4032880 con 0x7f5ed406fc80 2024-04-15T15:25:08.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.076+0000 7f5ecaa00640 1 -- 172.21.15.150:0/1847182489 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec4032a20 con 0x7f5ed406fc80 2024-04-15T15:25:08.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.076+0000 7f5ecaa00640 1 -- 172.21.15.150:0/1847182489 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4032e10 con 0x7f5ed406fc80 2024-04-15T15:25:08.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.076+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41f7eb0 con 0x7f5ed406fc80 2024-04-15T15:25:08.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.076+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed4142680 con 0x7f5ed406fc80 2024-04-15T15:25:08.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.077+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed4068c80 con 0x7f5ed406fc80 2024-04-15T15:25:08.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.078+0000 7f5ecaa00640 1 -- 172.21.15.150:0/1847182489 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ec40453f0 con 0x7f5ed406fc80 2024-04-15T15:25:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.079+0000 7f5ecaa00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5eb006c940 0x7f5eb006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:08.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.079+0000 7f5ecaa00640 1 -- 172.21.15.150:0/1847182489 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5ec40adc70 con 0x7f5ed406fc80 2024-04-15T15:25:08.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.080+0000 7f5ec9400640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5eb006c940 0x7f5eb006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:08.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.081+0000 7f5ec9400640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5eb006c940 0x7f5eb006ee00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc006fd0 tx=0x7f5ebc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:08.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.083+0000 7f5ecaa00640 1 -- 172.21.15.150:0/1847182489 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ec4040070 con 0x7f5ed406fc80 2024-04-15T15:25:08.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.327+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5ed40ce070 con 0x7f5ed406fc80 2024-04-15T15:25:08.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.931+0000 7f5ecaa00640 1 -- 172.21.15.150:0/1847182489 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f5ec407c110 con 0x7f5ed406fc80 2024-04-15T15:25:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5eb006c940 msgr2=0x7f5eb006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5eb006c940 0x7f5eb006ee00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc006fd0 tx=0x7f5ebc008040 comp rx=0 tx=0).stop 2024-04-15T15:25:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed406fc80 msgr2=0x7f5ed4142140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed406fc80 0x7f5ed4142140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5ec402b8c0 tx=0x7f5ec40046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 shutdown_connections 2024-04-15T15:25:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5eb006c940 0x7f5eb006ee00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ed415a440 0x7f5ed415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ed40cd620 0x7f5ed40cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 --2- 172.21.15.150:0/1847182489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed406fc80 0x7f5ed4142140 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 >> 172.21.15.150:0/1847182489 conn(0x7f5ed40b2460 msgr2=0x7f5ed415f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 shutdown_connections 2024-04-15T15:25:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:08.935+0000 7f5ed8c00640 1 -- 172.21.15.150:0/1847182489 wait complete. 2024-04-15T15:25:08.959 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-15T15:25:08.959 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-15T15:25:09.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.173+0000 7fcb63200640 1 Processor -- start 2024-04-15T15:25:09.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.173+0000 7fcb63200640 1 -- start start 2024-04-15T15:25:09.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.174+0000 7fcb63200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb64158f10 0x7fcb6406f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:09.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb63200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcb640b3dc0 0x7fcb6406f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:09.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb63200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb6415f230 con 0x7fcb640cd4e0 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb63200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fcb640ce040 con 0x7fcb640b3dc0 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb63200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6415f4d0 con 0x7fcb64158f10 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb5be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcb640b3dc0 0x7fcb6406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb5be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcb640b3dc0 0x7fcb6406f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55166/0 (socket says 172.21.15.150:55166) 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb5be00640 1 -- 172.21.15.150:0/2166295413 learned_addr learned my addr 172.21.15.150:0/2166295413 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb61e00640 1 --2- 172.21.15.150:0/2166295413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb64158f10 0x7fcb6406f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb5be00640 1 -- 172.21.15.150:0/2166295413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb64158f10 msgr2=0x7fcb6406f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb5be00640 1 --2- 172.21.15.150:0/2166295413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb64158f10 0x7fcb6406f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.175+0000 7fcb5be00640 1 -- 172.21.15.150:0/2166295413 >> v1:172.21.15.187:6789/0 conn(0x7fcb640cd4e0 legacy=0x7fcb640cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.176+0000 7fcb5be00640 1 -- 172.21.15.150:0/2166295413 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6415f7b0 con 0x7fcb640b3dc0 2024-04-15T15:25:09.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.176+0000 7fcb61e00640 1 --2- 172.21.15.150:0/2166295413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb64158f10 0x7fcb6406f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:09.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.177+0000 7fcb5be00640 1 --2- 172.21.15.150:0/2166295413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb640b3dc0 0x7fcb6406f570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009870 tx=0x7fcb4c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188f92ae91900cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:09.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.177+0000 7fcb61400640 1 -- 172.21.15.150:0/2166295413 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0317c0 con 0x7fcb640b3dc0 2024-04-15T15:25:09.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.177+0000 7fcb61400640 1 -- 172.21.15.150:0/2166295413 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4c037070 con 0x7fcb640b3dc0 2024-04-15T15:25:09.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.177+0000 7fcb61400640 1 -- 172.21.15.150:0/2166295413 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c031b80 con 0x7fcb640b3dc0 2024-04-15T15:25:09.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.178+0000 7fcb63200640 1 -- 172.21.15.150:0/2166295413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb640b3dc0 msgr2=0x7fcb6406f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:09.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.178+0000 7fcb63200640 1 --2- 172.21.15.150:0/2166295413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb640b3dc0 0x7fcb6406f570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009870 tx=0x7fcb4c02b860 comp rx=0 tx=0).stop 2024-04-15T15:25:09.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.178+0000 7fcb63200640 1 -- 172.21.15.150:0/2166295413 shutdown_connections 2024-04-15T15:25:09.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.178+0000 7fcb63200640 1 --2- 172.21.15.150:0/2166295413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb640b3dc0 0x7fcb6406f570 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.178+0000 7fcb63200640 1 --2- 172.21.15.150:0/2166295413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb64158f10 0x7fcb6406f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.178+0000 7fcb63200640 1 -- 172.21.15.150:0/2166295413 >> 172.21.15.150:0/2166295413 conn(0x7fcb6406b930 msgr2=0x7fcb6406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:09.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.178+0000 7fcb63200640 1 -- 172.21.15.150:0/2166295413 shutdown_connections 2024-04-15T15:25:09.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.178+0000 7fcb63200640 1 -- 172.21.15.150:0/2166295413 wait complete. 2024-04-15T15:25:09.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.180+0000 7fcb63200640 1 Processor -- start 2024-04-15T15:25:09.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.180+0000 7fcb63200640 1 -- start start 2024-04-15T15:25:09.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb63200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb640b3dc0 0x7fcb6414a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:09.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb63200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb640cd4e0 0x7fcb6414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:09.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb63200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb64158f10 0x7fcb6414e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:09.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb63200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641592f0 con 0x7fcb64158f10 2024-04-15T15:25:09.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb63200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6415fd70 con 0x7fcb640cd4e0 2024-04-15T15:25:09.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb63200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb64159470 con 0x7fcb640b3dc0 2024-04-15T15:25:09.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb62800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb64158f10 0x7fcb6414e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb62800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb64158f10 0x7fcb6414e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55180/0 (socket says 172.21.15.150:55180) 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb62800640 1 -- 172.21.15.150:0/4243117688 learned_addr learned my addr 172.21.15.150:0/4243117688 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb61e00640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb640b3dc0 0x7fcb6414a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb62800640 1 -- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb640b3dc0 msgr2=0x7fcb6414a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb62800640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb640b3dc0 0x7fcb6414a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb62800640 1 -- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb640cd4e0 msgr2=0x7fcb6414df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.181+0000 7fcb5be00640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb640cd4e0 0x7fcb6414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb62800640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb640cd4e0 0x7fcb6414df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb62800640 1 -- 172.21.15.150:0/4243117688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6414a990 con 0x7fcb64158f10 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb5be00640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb640cd4e0 0x7fcb6414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:09.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb62800640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb64158f10 0x7fcb6414e450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c780 tx=0x7fcb5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:09.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb5a000640 1 -- 172.21.15.150:0/4243117688 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c004b60 con 0x7fcb64158f10 2024-04-15T15:25:09.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb5a000640 1 -- 172.21.15.150:0/4243117688 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5c004d00 con 0x7fcb64158f10 2024-04-15T15:25:09.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb5a000640 1 -- 172.21.15.150:0/4243117688 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00aa90 con 0x7fcb64158f10 2024-04-15T15:25:09.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb641549c0 con 0x7fcb64158f10 2024-04-15T15:25:09.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.182+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6414ac50 con 0x7fcb64158f10 2024-04-15T15:25:09.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.183+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb640755c0 con 0x7fcb64158f10 2024-04-15T15:25:09.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.184+0000 7fcb5a000640 1 -- 172.21.15.150:0/4243117688 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb5c00ac30 con 0x7fcb64158f10 2024-04-15T15:25:09.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.185+0000 7fcb5a000640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:09.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.185+0000 7fcb5a000640 1 -- 172.21.15.150:0/4243117688 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fcb5c08be70 con 0x7fcb64158f10 2024-04-15T15:25:09.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.186+0000 7fcb61e00640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:09.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.187+0000 7fcb61e00640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb4006c8f0 0x7fcb4006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b840 tx=0x7fcb54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:09.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.190+0000 7fcb5a000640 1 -- 172.21.15.150:0/4243117688 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb5c020070 con 0x7fcb64158f10 2024-04-15T15:25:09.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.433+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcb641554e0 con 0x7fcb64158f10 2024-04-15T15:25:09.784 INFO:tasks.ceph.mds.a.smithi150.stderr:2024-04-15T15:25:09.782+0000 7fad61200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:25:09.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.941+0000 7fcb5a000640 1 -- 172.21.15.150:0/4243117688 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fcb5c05a2a0 con 0x7fcb64158f10 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb4006c8f0 msgr2=0x7fcb4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb4006c8f0 0x7fcb4006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b840 tx=0x7fcb54008040 comp rx=0 tx=0).stop 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb64158f10 msgr2=0x7fcb6414e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb64158f10 0x7fcb6414e450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c780 tx=0x7fcb5c00cc40 comp rx=0 tx=0).stop 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 shutdown_connections 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb64158f10 0x7fcb6414e450 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb640cd4e0 0x7fcb6414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcb4006c8f0 0x7fcb4006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 --2- 172.21.15.150:0/4243117688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb640b3dc0 0x7fcb6414a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 >> 172.21.15.150:0/4243117688 conn(0x7fcb6406b930 msgr2=0x7fcb640b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 shutdown_connections 2024-04-15T15:25:09.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:09.946+0000 7fcb63200640 1 -- 172.21.15.150:0/4243117688 wait complete. 2024-04-15T15:25:09.972 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-15T15:25:09.973 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:10.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.180+0000 7f28c6600640 1 Processor -- start 2024-04-15T15:25:10.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.180+0000 7f28c6600640 1 -- start start 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.182+0000 7f28c6600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c01461f0 0x7f28c00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.182+0000 7f28c6600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f28c00cedf0 0x7f28c00b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.182+0000 7f28c6600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c00b4be0 con 0x7f28c00b3e10 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.182+0000 7f28c6600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f28c00a9640 con 0x7f28c00cedf0 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.182+0000 7f28c6600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28c00cf250 con 0x7f28c01461f0 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.182+0000 7f28b7e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f28c00cedf0 0x7f28c00b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.182+0000 7f28bfe00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f28c00b3e10 0x7f28c00cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:55160/0 (socket says 172.21.15.150:55160) 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.182+0000 7f28bfe00640 1 -- 172.21.15.150:0/2989628321 learned_addr learned my addr 172.21.15.150:0/2989628321 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:10.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.183+0000 7f28bf400640 1 --2- 172.21.15.150:0/2989628321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c01461f0 0x7f28c00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.183+0000 7f28b7e00640 1 -- 172.21.15.150:0/2989628321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c01461f0 msgr2=0x7f28c00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:10.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.183+0000 7f28b7e00640 1 --2- 172.21.15.150:0/2989628321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c01461f0 0x7f28c00ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.183+0000 7f28b7e00640 1 -- 172.21.15.150:0/2989628321 >> v1:172.21.15.150:6789/0 conn(0x7f28c00b3e10 legacy=0x7f28c00cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.183+0000 7f28b7e00640 1 -- 172.21.15.150:0/2989628321 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c00b4620 con 0x7f28c00cedf0 2024-04-15T15:25:10.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.184+0000 7f28b7e00640 1 --2- 172.21.15.150:0/2989628321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00b38d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f28a80097d0 tx=0x7f28a802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7d17464b123b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:10.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.184+0000 7f28bea00640 1 -- 172.21.15.150:0/2989628321 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8035020 con 0x7f28c00cedf0 2024-04-15T15:25:10.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.184+0000 7f28bea00640 1 -- 172.21.15.150:0/2989628321 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a8031810 con 0x7f28c00cedf0 2024-04-15T15:25:10.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.184+0000 7f28bea00640 1 -- 172.21.15.150:0/2989628321 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8031be0 con 0x7f28c00cedf0 2024-04-15T15:25:10.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.185+0000 7f28c6600640 1 -- 172.21.15.150:0/2989628321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 msgr2=0x7f28c00b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.185+0000 7f28c6600640 1 --2- 172.21.15.150:0/2989628321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00b38d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f28a80097d0 tx=0x7f28a802b8b0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.185+0000 7f28c6600640 1 -- 172.21.15.150:0/2989628321 shutdown_connections 2024-04-15T15:25:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.185+0000 7f28c6600640 1 --2- 172.21.15.150:0/2989628321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00b38d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.185+0000 7f28c6600640 1 --2- 172.21.15.150:0/2989628321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c01461f0 0x7f28c00ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.185+0000 7f28c6600640 1 -- 172.21.15.150:0/2989628321 >> 172.21.15.150:0/2989628321 conn(0x7f28c00b27b0 msgr2=0x7f28c00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.185+0000 7f28c6600640 1 -- 172.21.15.150:0/2989628321 shutdown_connections 2024-04-15T15:25:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.185+0000 7f28c6600640 1 -- 172.21.15.150:0/2989628321 wait complete. 2024-04-15T15:25:10.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.187+0000 7f28c6600640 1 Processor -- start 2024-04-15T15:25:10.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.187+0000 7f28c6600640 1 -- start start 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28c6600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b3e10 0x7f28c015c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28c6600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28c6600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c01461f0 0x7f28c00ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28c6600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f28c015d290 con 0x7f28c00cedf0 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28c6600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28c015cfb0 con 0x7f28c01461f0 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28c6600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28c01465d0 con 0x7f28c00b3e10 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28b7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28b7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00abdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55208/0 (socket says 172.21.15.150:55208) 2024-04-15T15:25:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.188+0000 7f28b7e00640 1 -- 172.21.15.150:0/1380651749 learned_addr learned my addr 172.21.15.150:0/1380651749 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28bf400640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b3e10 0x7f28c015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28bfe00640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c01461f0 0x7f28c00ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28b7e00640 1 -- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b3e10 msgr2=0x7f28c015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28b7e00640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b3e10 0x7f28c015c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28b7e00640 1 -- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c01461f0 msgr2=0x7f28c00ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28b7e00640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c01461f0 0x7f28c00ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28b7e00640 1 -- 172.21.15.150:0/1380651749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c01e7600 con 0x7f28c00cedf0 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28bf400640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b3e10 0x7f28c015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28bfe00640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c01461f0 0x7f28c00ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28b7e00640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00abdc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f28a80098c0 tx=0x7f28a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.189+0000 7f28b7400640 1 -- 172.21.15.150:0/1380651749 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8041070 con 0x7f28c00cedf0 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.190+0000 7f28b7400640 1 -- 172.21.15.150:0/1380651749 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a803f070 con 0x7f28c00cedf0 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.190+0000 7f28b7400640 1 -- 172.21.15.150:0/1380651749 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8035040 con 0x7f28c00cedf0 2024-04-15T15:25:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.190+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c01e78e0 con 0x7f28c00cedf0 2024-04-15T15:25:10.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.190+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c00ae330 con 0x7f28c00cedf0 2024-04-15T15:25:10.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.191+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c00d0000 con 0x7f28c00cedf0 2024-04-15T15:25:10.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.192+0000 7f28b7400640 1 -- 172.21.15.150:0/1380651749 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28a8038020 con 0x7f28c00cedf0 2024-04-15T15:25:10.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.193+0000 7f28b7400640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f289c06c8f0 0x7f289c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.193+0000 7f28b7400640 1 -- 172.21.15.150:0/1380651749 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f28a80ad990 con 0x7f28c00cedf0 2024-04-15T15:25:10.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.193+0000 7f28bf400640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f289c06c8f0 0x7f289c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.195+0000 7f28bf400640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f289c06c8f0 0x7f289c06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006fd0 tx=0x7f28b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:10.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.197+0000 7f28b7400640 1 -- 172.21.15.150:0/1380651749 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28c00d0000 con 0x7f28c00cedf0 2024-04-15T15:25:10.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.429+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28c0068c80 con 0x7f28c00cedf0 2024-04-15T15:25:10.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.430+0000 7f28b7400640 1 -- 172.21.15.150:0/1380651749 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f28a807b6d0 con 0x7f28c00cedf0 2024-04-15T15:25:10.432 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:10.432 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:09.435623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6835/3327940880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3327940880},{"type":"v1","addr":"172.21.15.150:6835","nonce":3327940880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-15T15:25:10.433 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 17 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f289c06c8f0 msgr2=0x7f289c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f289c06c8f0 0x7f289c06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006fd0 tx=0x7f28b0008040 comp rx=0 tx=0).stop 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 msgr2=0x7f28c00abdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00abdc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f28a80098c0 tx=0x7f28a80046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 shutdown_connections 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c01461f0 0x7f28c00ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00cedf0 0x7f28c00abdc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f289c06c8f0 0x7f289c06edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 --2- 172.21.15.150:0/1380651749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b3e10 0x7f28c015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 >> 172.21.15.150:0/1380651749 conn(0x7f28c00b27b0 msgr2=0x7f28c00b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:10.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 shutdown_connections 2024-04-15T15:25:10.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.433+0000 7f28c6600640 1 -- 172.21.15.150:0/1380651749 wait complete. 2024-04-15T15:25:10.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:10.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.658+0000 7f4f6fe00640 1 Processor -- start 2024-04-15T15:25:10.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.658+0000 7f4f6fe00640 1 -- start start 2024-04-15T15:25:10.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.660+0000 7f4f6fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7015b480 0x7f4f700cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.660+0000 7f4f6fe00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7015f2b0 con 0x7f4f7015b860 2024-04-15T15:25:10.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.660+0000 7f4f6fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7015f590 con 0x7f4f700cdde0 2024-04-15T15:25:10.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.660+0000 7f4f6fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7015f060 con 0x7f4f7015b480 2024-04-15T15:25:10.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.660+0000 7f4f6f400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f4f7015b860 0x7f4f700b3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:55172/0 (socket says 172.21.15.150:55172) 2024-04-15T15:25:10.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.660+0000 7f4f6f400640 1 -- 172.21.15.150:0/2051205212 learned_addr learned my addr 172.21.15.150:0/2051205212 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:10.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.660+0000 7f4f6ea00640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7015b480 0x7f4f700cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182466516 0 0) 0x7f4f7015f2b0 con 0x7f4f7015b860 2024-04-15T15:25:10.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f54003520 con 0x7f4f7015b860 2024-04-15T15:25:10.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3262418926 0 0) 0x7f4f54003520 con 0x7f4f7015b860 2024-04-15T15:25:10.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7015b480 msgr2=0x7f4f700cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:10.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7015b480 0x7f4f700cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 >> v1:172.21.15.187:6789/0 conn(0x7f4f700cdde0 legacy=0x7f4f7006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7015f870 con 0x7f4f7015b860 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f4f680019f0 con 0x7f4f7015b860 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 >> v1:172.21.15.150:6789/0 conn(0x7f4f7015b860 legacy=0x7f4f700b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7015b480 0x7f4f540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f54003f60 0x7f4f54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7015f060 con 0x7f4f54004d40 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5400dcb0 con 0x7f4f54003f60 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.661+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5400deb0 con 0x7f4f7015b480 2024-04-15T15:25:10.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6f400640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6e000640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f54003f60 0x7f4f54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6ea00640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7015b480 0x7f4f540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6f400640 1 -- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7015b480 msgr2=0x7f4f540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:10.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6f400640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7015b480 0x7f4f540039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6f400640 1 -- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f54003f60 msgr2=0x7f4f54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:10.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6f400640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f54003f60 0x7f4f54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6f400640 1 -- 172.21.15.150:0/2051205212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f54003520 con 0x7f4f54004d40 2024-04-15T15:25:10.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.662+0000 7f4f6e000640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f54003f60 0x7f4f54004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.663+0000 7f4f6f400640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f540051c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4f7015bc40 tx=0x7f4f54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154134bc02d5c9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800a460 con 0x7f4f54004d40 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f68034650 con 0x7f4f54004d40 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6d600640 1 -- 172.21.15.150:0/2051205212 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68034a80 con 0x7f4f54004d40 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6fe00640 1 -- 172.21.15.150:0/2051205212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 msgr2=0x7f4f540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6fe00640 1 --2- 172.21.15.150:0/2051205212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f540051c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4f7015bc40 tx=0x7f4f54005830 comp rx=0 tx=0).stop 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6f400640 1 -- 172.21.15.150:0/2051205212 reap_dead start 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6fe00640 1 -- 172.21.15.150:0/2051205212 shutdown_connections 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6fe00640 1 -- 172.21.15.150:0/2051205212 >> 172.21.15.150:0/2051205212 conn(0x7f4f7006b930 msgr2=0x7f4f7006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:10.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.664+0000 7f4f6fe00640 1 -- 172.21.15.150:0/2051205212 shutdown_connections 2024-04-15T15:25:10.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.665+0000 7f4f6fe00640 1 -- 172.21.15.150:0/2051205212 wait complete. 2024-04-15T15:25:10.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.666+0000 7f4f6fe00640 1 Processor -- start 2024-04-15T15:25:10.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.666+0000 7f4f6fe00640 1 -- start start 2024-04-15T15:25:10.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.667+0000 7f4f6fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f700b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.667+0000 7f4f6fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7006f000 0x7f4f7014af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.667+0000 7f4f6fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f7014bc80 0x7f4f701584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.667+0000 7f4f6fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f700cd550 con 0x7f4f54004d40 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.667+0000 7f4f6fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7015fec0 con 0x7f4f7014bc80 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.667+0000 7f4f6fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7006f4f0 con 0x7f4f7006f000 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.667+0000 7f4f6ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f700b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f700b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55226/0 (socket says 172.21.15.150:55226) 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6ea00640 1 -- 172.21.15.150:0/3270439200 learned_addr learned my addr 172.21.15.150:0/3270439200 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6f400640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f7014bc80 0x7f4f701584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6e000640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7006f000 0x7f4f7014af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6ea00640 1 -- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7006f000 msgr2=0x7f4f7014af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6ea00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7006f000 0x7f4f7014af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6ea00640 1 -- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f7014bc80 msgr2=0x7f4f701584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6ea00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f7014bc80 0x7f4f701584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6ea00640 1 -- 172.21.15.150:0/3270439200 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f700b20c0 con 0x7f4f54004d40 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6f400640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f7014bc80 0x7f4f701584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.668+0000 7f4f6ea00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f700b1b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4f60011600 tx=0x7f4f60011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.669+0000 7f4f66a00640 1 -- 172.21.15.150:0/3270439200 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000f920 con 0x7f4f54004d40 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.669+0000 7f4f66a00640 1 -- 172.21.15.150:0/3270439200 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6000fac0 con 0x7f4f54004d40 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.669+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f700ca5b0 con 0x7f4f54004d40 2024-04-15T15:25:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.669+0000 7f4f66a00640 1 -- 172.21.15.150:0/3270439200 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000fe70 con 0x7f4f54004d40 2024-04-15T15:25:10.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.669+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f700cab70 con 0x7f4f54004d40 2024-04-15T15:25:10.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.670+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f700755c0 con 0x7f4f54004d40 2024-04-15T15:25:10.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.671+0000 7f4f66a00640 1 -- 172.21.15.150:0/3270439200 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f60015460 con 0x7f4f54004d40 2024-04-15T15:25:10.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.672+0000 7f4f66a00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f5006c8f0 0x7f4f5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:10.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.672+0000 7f4f66a00640 1 -- 172.21.15.150:0/3270439200 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f4f6008c640 con 0x7f4f54004d40 2024-04-15T15:25:10.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.672+0000 7f4f6e000640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f5006c8f0 0x7f4f5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:10.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.673+0000 7f4f6e000640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f5006c8f0 0x7f4f5006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c007fd0 tx=0x7f4f5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:10.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.676+0000 7f4f66a00640 1 -- 172.21.15.150:0/3270439200 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f60019030 con 0x7f4f54004d40 2024-04-15T15:25:10.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.875+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f70156b90 con 0x7f4f54004d40 2024-04-15T15:25:10.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.877+0000 7f4f66a00640 1 -- 172.21.15.150:0/3270439200 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7f4f6005a940 con 0x7f4f54004d40 2024-04-15T15:25:10.879 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:10.879 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":35,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:07.917174+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:10.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f5006c8f0 msgr2=0x7f4f5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f5006c8f0 0x7f4f5006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c007fd0 tx=0x7f4f5c009040 comp rx=0 tx=0).stop 2024-04-15T15:25:10.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 msgr2=0x7f4f700b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:10.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f700b1b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4f60011600 tx=0x7f4f60011ac0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 shutdown_connections 2024-04-15T15:25:10.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f7014bc80 0x7f4f701584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f5006c8f0 0x7f4f5006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f7006f000 0x7f4f7014af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 --2- 172.21.15.150:0/3270439200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f54004d40 0x7f4f700b1b80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:10.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 >> 172.21.15.150:0/3270439200 conn(0x7f4f7006b930 msgr2=0x7f4f700b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:10.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 shutdown_connections 2024-04-15T15:25:10.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:10.880+0000 7f4f6fe00640 1 -- 172.21.15.150:0/3270439200 wait complete. 2024-04-15T15:25:10.902 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-15T15:25:10.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:25:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.111+0000 7fb9a5400640 1 Processor -- start 2024-04-15T15:25:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.111+0000 7fb9a5400640 1 -- start start 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb9a5400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb9a00cee00 0x7fb9a006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb9a5400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb9a0071f50 0x7fb9a00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb9a5400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a015cbe0 0x7fb9a015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb9a5400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a006f4f0 con 0x7fb9a00cee00 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb9a5400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a00728f0 con 0x7fb9a0071f50 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb9a5400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a015dbb0 con 0x7fb9a015cbe0 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb99fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb9a00cee00 0x7fb9a006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb99fe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb9a00cee00 0x7fb9a006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55234/0 (socket says 172.21.15.150:55234) 2024-04-15T15:25:11.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb99fe00640 1 -- 172.21.15.150:0/4095097711 learned_addr learned my addr 172.21.15.150:0/4095097711 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:11.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.113+0000 7fb9a4a00640 1 --2- 172.21.15.150:0/4095097711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a015cbe0 0x7fb9a015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.114+0000 7fb997400640 1 --2- 172.21.15.150:0/4095097711 >> v2:172.21.15.187:3300/0 conn(0x7fb9a0071f50 0x7fb9a00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.114+0000 7fb99fe00640 1 -- 172.21.15.150:0/4095097711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a015cbe0 msgr2=0x7fb9a015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:11.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.114+0000 7fb99fe00640 1 --2- 172.21.15.150:0/4095097711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a015cbe0 0x7fb9a015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.114+0000 7fb99fe00640 1 -- 172.21.15.150:0/4095097711 >> v2:172.21.15.187:3300/0 conn(0x7fb9a0071f50 msgr2=0x7fb9a00723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:11.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.114+0000 7fb99fe00640 1 --2- 172.21.15.150:0/4095097711 >> v2:172.21.15.187:3300/0 conn(0x7fb9a0071f50 0x7fb9a00723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.114+0000 7fb99fe00640 1 -- 172.21.15.150:0/4095097711 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a015d8d0 con 0x7fb9a00cee00 2024-04-15T15:25:11.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.116+0000 7fb99fe00640 1 --2- 172.21.15.150:0/4095097711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a006e8c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b5aa0 tx=0x7fb99002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1175caae52fb165 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:11.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.116+0000 7fb99f400640 1 -- 172.21.15.150:0/4095097711 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990035020 con 0x7fb9a00cee00 2024-04-15T15:25:11.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.116+0000 7fb99f400640 1 -- 172.21.15.150:0/4095097711 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb990031640 con 0x7fb9a00cee00 2024-04-15T15:25:11.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.116+0000 7fb99f400640 1 -- 172.21.15.150:0/4095097711 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99002d040 con 0x7fb9a00cee00 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 -- 172.21.15.150:0/4095097711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 msgr2=0x7fb9a006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 --2- 172.21.15.150:0/4095097711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a006e8c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b5aa0 tx=0x7fb99002b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 -- 172.21.15.150:0/4095097711 shutdown_connections 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 --2- 172.21.15.150:0/4095097711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a015cbe0 0x7fb9a015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 --2- 172.21.15.150:0/4095097711 >> v2:172.21.15.187:3300/0 conn(0x7fb9a0071f50 0x7fb9a00723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 --2- 172.21.15.150:0/4095097711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a006e8c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 -- 172.21.15.150:0/4095097711 >> 172.21.15.150:0/4095097711 conn(0x7fb9a006b930 msgr2=0x7fb9a006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 -- 172.21.15.150:0/4095097711 shutdown_connections 2024-04-15T15:25:11.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.117+0000 7fb9a5400640 1 -- 172.21.15.150:0/4095097711 wait complete. 2024-04-15T15:25:11.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.119+0000 7fb9a5400640 1 Processor -- start 2024-04-15T15:25:11.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.119+0000 7fb9a5400640 1 -- start start 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb9a5400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a0071f50 0x7fb9a01f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb9a5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a00b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb9a5400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a015cbe0 0x7fb9a00b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb9a5400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a006e3a0 con 0x7fb9a00cee00 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb9a5400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a006e0c0 con 0x7fb9a015cbe0 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb9a5400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a015d5b0 con 0x7fb9a0071f50 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb997400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a00b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb997400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a00b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55238/0 (socket says 172.21.15.150:55238) 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.120+0000 7fb997400640 1 -- 172.21.15.150:0/3072746700 learned_addr learned my addr 172.21.15.150:0/3072746700 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb99fe00640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a0071f50 0x7fb9a01f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb997400640 1 -- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a0071f50 msgr2=0x7fb9a01f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb9a4a00640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a015cbe0 0x7fb9a00b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb997400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a0071f50 0x7fb9a01f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb997400640 1 -- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a015cbe0 msgr2=0x7fb9a00b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb997400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a015cbe0 0x7fb9a00b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb997400640 1 -- 172.21.15.150:0/3072746700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a01f7bc0 con 0x7fb9a00cee00 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb997400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a00b3e90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb98800d5f0 tx=0x7fb98800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb99cc00640 1 -- 172.21.15.150:0/3072746700 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9880125f0 con 0x7fb9a00cee00 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb99cc00640 1 -- 172.21.15.150:0/3072746700 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb988019070 con 0x7fb9a00cee00 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a00b4930 con 0x7fb9a00cee00 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb99cc00640 1 -- 172.21.15.150:0/3072746700 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9880129d0 con 0x7fb9a00cee00 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.121+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a00b4ef0 con 0x7fb9a00cee00 2024-04-15T15:25:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.122+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a00755c0 con 0x7fb9a00cee00 2024-04-15T15:25:11.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.123+0000 7fb99cc00640 1 -- 172.21.15.150:0/3072746700 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb988012b70 con 0x7fb9a00cee00 2024-04-15T15:25:11.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.124+0000 7fb99cc00640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb97c06c8f0 0x7fb97c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.124+0000 7fb99cc00640 1 -- 172.21.15.150:0/3072746700 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb98808bfd0 con 0x7fb9a00cee00 2024-04-15T15:25:11.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.125+0000 7fb99fe00640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb97c06c8f0 0x7fb97c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.126+0000 7fb99fe00640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb97c06c8f0 0x7fb97c06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb990036000 tx=0x7fb990008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:11.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.129+0000 7fb99cc00640 1 -- 172.21.15.150:0/3072746700 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb98805a360 con 0x7fb9a00cee00 2024-04-15T15:25:11.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.325+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9a0156c50 con 0x7fb97c06c8f0 2024-04-15T15:25:11.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.346+0000 7fb99cc00640 1 -- 172.21.15.150:0/3072746700 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7fb9a0156c50 con 0x7fb97c06c8f0 2024-04-15T15:25:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.352+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb97c06c8f0 msgr2=0x7fb97c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.352+0000 7fb9a5400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb97c06c8f0 0x7fb97c06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb990036000 tx=0x7fb990008030 comp rx=0 tx=0).stop 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.352+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 msgr2=0x7fb9a00b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.352+0000 7fb9a5400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a00b3e90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb98800d5f0 tx=0x7fb98800dab0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.353+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 shutdown_connections 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.353+0000 7fb9a5400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9a015cbe0 0x7fb9a00b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.353+0000 7fb9a5400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9a00cee00 0x7fb9a00b3e90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.353+0000 7fb9a5400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb97c06c8f0 0x7fb97c06edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.353+0000 7fb9a5400640 1 --2- 172.21.15.150:0/3072746700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9a0071f50 0x7fb9a01f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.353+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 >> 172.21.15.150:0/3072746700 conn(0x7fb9a006b930 msgr2=0x7fb9a015f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.353+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 shutdown_connections 2024-04-15T15:25:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.353+0000 7fb9a5400640 1 -- 172.21.15.150:0/3072746700 wait complete. 2024-04-15T15:25:11.375 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-15T15:25:11.243224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220356,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754364,"statfs":{"total":773094113280,"available":772868468736,"internally_reserved":0,"allocated":4268032,"data_stored":2353604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002162"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111987+0000","last_change":"2024-04-15T15:25:06.937502+0000","last_active":"2024-04-15T15:25:08.111987+0000","last_peered":"2024-04-15T15:25:08.111987+0000","last_clean":"2024-04-15T15:25:08.111987+0000","last_became_active":"2024-04-15T15:25:06.933540+0000","last_became_peered":"2024-04-15T15:25:06.933540+0000","last_unstale":"2024-04-15T15:25:08.111987+0000","last_undegraded":"2024-04-15T15:25:08.111987+0000","last_fullsized":"2024-04-15T15:25:08.111987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:25:18.723432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109742+0000","last_change":"2024-04-15T15:25:06.928928+0000","last_active":"2024-04-15T15:25:08.109742+0000","last_peered":"2024-04-15T15:25:08.109742+0000","last_clean":"2024-04-15T15:25:08.109742+0000","last_became_active":"2024-04-15T15:25:06.928240+0000","last_became_peered":"2024-04-15T15:25:06.928240+0000","last_unstale":"2024-04-15T15:25:08.109742+0000","last_undegraded":"2024-04-15T15:25:08.109742+0000","last_fullsized":"2024-04-15T15:25:08.109742+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:03.905713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.940242+0000","last_change":"2024-04-15T15:25:06.935862+0000","last_active":"2024-04-15T15:25:07.940242+0000","last_peered":"2024-04-15T15:25:07.940242+0000","last_clean":"2024-04-15T15:25:07.940242+0000","last_became_active":"2024-04-15T15:25:06.934512+0000","last_became_peered":"2024-04-15T15:25:06.934512+0000","last_unstale":"2024-04-15T15:25:07.940242+0000","last_undegraded":"2024-04-15T15:25:07.940242+0000","last_fullsized":"2024-04-15T15:25:07.940242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:43:24.414025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933144+0000","last_change":"2024-04-15T15:25:06.926921+0000","last_active":"2024-04-15T15:25:07.933144+0000","last_peered":"2024-04-15T15:25:07.933144+0000","last_clean":"2024-04-15T15:25:07.933144+0000","last_became_active":"2024-04-15T15:25:06.926636+0000","last_became_peered":"2024-04-15T15:25:06.926636+0000","last_unstale":"2024-04-15T15:25:07.933144+0000","last_undegraded":"2024-04-15T15:25:07.933144+0000","last_fullsized":"2024-04-15T15:25:07.933144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:31:25.313430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.935015+0000","last_change":"2024-04-15T15:25:06.929911+0000","last_active":"2024-04-15T15:25:06.935015+0000","last_peered":"2024-04-15T15:25:06.935015+0000","last_clean":"2024-04-15T15:25:06.935015+0000","last_became_active":"2024-04-15T15:25:06.926916+0000","last_became_peered":"2024-04-15T15:25:06.926916+0000","last_unstale":"2024-04-15T15:25:06.935015+0000","last_undegraded":"2024-04-15T15:25:06.935015+0000","last_fullsized":"2024-04-15T15:25:06.935015+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:48:00.129752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.108910+0000","last_change":"2024-04-15T15:25:06.928707+0000","last_active":"2024-04-15T15:25:08.108910+0000","last_peered":"2024-04-15T15:25:08.108910+0000","last_clean":"2024-04-15T15:25:08.108910+0000","last_became_active":"2024-04-15T15:25:06.928420+0000","last_became_peered":"2024-04-15T15:25:06.928420+0000","last_unstale":"2024-04-15T15:25:08.108910+0000","last_undegraded":"2024-04-15T15:25:08.108910+0000","last_fullsized":"2024-04-15T15:25:08.108910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:23:14.407218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938534+0000","last_change":"2024-04-15T15:25:06.936038+0000","last_active":"2024-04-15T15:25:07.938534+0000","last_peered":"2024-04-15T15:25:07.938534+0000","last_clean":"2024-04-15T15:25:07.938534+0000","last_became_active":"2024-04-15T15:25:06.935691+0000","last_became_peered":"2024-04-15T15:25:06.935691+0000","last_unstale":"2024-04-15T15:25:07.938534+0000","last_undegraded":"2024-04-15T15:25:07.938534+0000","last_fullsized":"2024-04-15T15:25:07.938534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:45:23.858988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937225+0000","last_change":"2024-04-15T15:25:06.933544+0000","last_active":"2024-04-15T15:25:07.937225+0000","last_peered":"2024-04-15T15:25:07.937225+0000","last_clean":"2024-04-15T15:25:07.937225+0000","last_became_active":"2024-04-15T15:25:06.933247+0000","last_became_peered":"2024-04-15T15:25:06.933247+0000","last_unstale":"2024-04-15T15:25:07.937225+0000","last_undegraded":"2024-04-15T15:25:07.937225+0000","last_fullsized":"2024-04-15T15:25:07.937225+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:12.196977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111314+0000","last_change":"2024-04-15T15:25:06.927357+0000","last_active":"2024-04-15T15:25:08.111314+0000","last_peered":"2024-04-15T15:25:08.111314+0000","last_clean":"2024-04-15T15:25:08.111314+0000","last_became_active":"2024-04-15T15:25:06.927034+0000","last_became_peered":"2024-04-15T15:25:06.927034+0000","last_unstale":"2024-04-15T15:25:08.111314+0000","last_undegraded":"2024-04-15T15:25:08.111314+0000","last_fullsized":"2024-04-15T15:25:08.111314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:42.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933865+0000","last_change":"2024-04-15T15:25:06.927238+0000","last_active":"2024-04-15T15:25:07.933865+0000","last_peered":"2024-04-15T15:25:07.933865+0000","last_clean":"2024-04-15T15:25:07.933865+0000","last_became_active":"2024-04-15T15:25:06.926927+0000","last_became_peered":"2024-04-15T15:25:06.926927+0000","last_unstale":"2024-04-15T15:25:07.933865+0000","last_undegraded":"2024-04-15T15:25:07.933865+0000","last_fullsized":"2024-04-15T15:25:07.933865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:44:23.747614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938361+0000","last_change":"2024-04-15T15:25:06.941742+0000","last_active":"2024-04-15T15:25:07.938361+0000","last_peered":"2024-04-15T15:25:07.938361+0000","last_clean":"2024-04-15T15:25:07.938361+0000","last_became_active":"2024-04-15T15:25:06.941405+0000","last_became_peered":"2024-04-15T15:25:06.941405+0000","last_unstale":"2024-04-15T15:25:07.938361+0000","last_undegraded":"2024-04-15T15:25:07.938361+0000","last_fullsized":"2024-04-15T15:25:07.938361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:43:24.414025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939355+0000","last_change":"2024-04-15T15:25:06.921853+0000","last_active":"2024-04-15T15:25:07.939355+0000","last_peered":"2024-04-15T15:25:07.939355+0000","last_clean":"2024-04-15T15:25:07.939355+0000","last_became_active":"2024-04-15T15:25:06.921491+0000","last_became_peered":"2024-04-15T15:25:06.921491+0000","last_unstale":"2024-04-15T15:25:07.939355+0000","last_undegraded":"2024-04-15T15:25:07.939355+0000","last_fullsized":"2024-04-15T15:25:07.939355+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:23:14.407218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.930089+0000","last_change":"2024-04-15T15:25:06.923195+0000","last_active":"2024-04-15T15:25:06.930089+0000","last_peered":"2024-04-15T15:25:06.930089+0000","last_clean":"2024-04-15T15:25:06.930089+0000","last_became_active":"2024-04-15T15:25:06.922872+0000","last_became_peered":"2024-04-15T15:25:06.922872+0000","last_unstale":"2024-04-15T15:25:06.930089+0000","last_undegraded":"2024-04-15T15:25:06.930089+0000","last_fullsized":"2024-04-15T15:25:06.930089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:13:33.617721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.108990+0000","last_change":"2024-04-15T15:25:06.924540+0000","last_active":"2024-04-15T15:25:08.108990+0000","last_peered":"2024-04-15T15:25:08.108990+0000","last_clean":"2024-04-15T15:25:08.108990+0000","last_became_active":"2024-04-15T15:25:06.924254+0000","last_became_peered":"2024-04-15T15:25:06.924254+0000","last_unstale":"2024-04-15T15:25:08.108990+0000","last_undegraded":"2024-04-15T15:25:08.108990+0000","last_fullsized":"2024-04-15T15:25:08.108990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:54.080186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939392+0000","last_change":"2024-04-15T15:25:06.927463+0000","last_active":"2024-04-15T15:25:07.939392+0000","last_peered":"2024-04-15T15:25:07.939392+0000","last_clean":"2024-04-15T15:25:07.939392+0000","last_became_active":"2024-04-15T15:25:06.927137+0000","last_became_peered":"2024-04-15T15:25:06.927137+0000","last_unstale":"2024-04-15T15:25:07.939392+0000","last_undegraded":"2024-04-15T15:25:07.939392+0000","last_fullsized":"2024-04-15T15:25:07.939392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:54.080186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938919+0000","last_change":"2024-04-15T15:25:06.940300+0000","last_active":"2024-04-15T15:25:07.938919+0000","last_peered":"2024-04-15T15:25:07.938919+0000","last_clean":"2024-04-15T15:25:07.938919+0000","last_became_active":"2024-04-15T15:25:06.940010+0000","last_became_peered":"2024-04-15T15:25:06.940010+0000","last_unstale":"2024-04-15T15:25:07.938919+0000","last_undegraded":"2024-04-15T15:25:07.938919+0000","last_fullsized":"2024-04-15T15:25:07.938919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:11:56.076782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938816+0000","last_change":"2024-04-15T15:25:06.939369+0000","last_active":"2024-04-15T15:25:07.938816+0000","last_peered":"2024-04-15T15:25:07.938816+0000","last_clean":"2024-04-15T15:25:07.938816+0000","last_became_active":"2024-04-15T15:25:06.938992+0000","last_became_peered":"2024-04-15T15:25:06.938992+0000","last_unstale":"2024-04-15T15:25:07.938816+0000","last_undegraded":"2024-04-15T15:25:07.938816+0000","last_fullsized":"2024-04-15T15:25:07.938816+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:12.118170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109743+0000","last_change":"2024-04-15T15:25:06.937257+0000","last_active":"2024-04-15T15:25:08.109743+0000","last_peered":"2024-04-15T15:25:08.109743+0000","last_clean":"2024-04-15T15:25:08.109743+0000","last_became_active":"2024-04-15T15:25:06.936982+0000","last_became_peered":"2024-04-15T15:25:06.936982+0000","last_unstale":"2024-04-15T15:25:08.109743+0000","last_undegraded":"2024-04-15T15:25:08.109743+0000","last_fullsized":"2024-04-15T15:25:08.109743+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:51.313422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109171+0000","last_change":"2024-04-15T15:25:06.936124+0000","last_active":"2024-04-15T15:25:08.109171+0000","last_peered":"2024-04-15T15:25:08.109171+0000","last_clean":"2024-04-15T15:25:08.109171+0000","last_became_active":"2024-04-15T15:25:06.931919+0000","last_became_peered":"2024-04-15T15:25:06.931919+0000","last_unstale":"2024-04-15T15:25:08.109171+0000","last_undegraded":"2024-04-15T15:25:08.109171+0000","last_fullsized":"2024-04-15T15:25:08.109171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:12.196977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.919464+0000","last_change":"2024-04-15T15:25:06.909188+0000","last_active":"2024-04-15T15:25:06.919464+0000","last_peered":"2024-04-15T15:25:06.919464+0000","last_clean":"2024-04-15T15:25:06.919464+0000","last_became_active":"2024-04-15T15:25:06.908430+0000","last_became_peered":"2024-04-15T15:25:06.908430+0000","last_unstale":"2024-04-15T15:25:06.919464+0000","last_undegraded":"2024-04-15T15:25:06.919464+0000","last_fullsized":"2024-04-15T15:25:06.919464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:01:01.235779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110489+0000","last_change":"2024-04-15T15:25:06.926364+0000","last_active":"2024-04-15T15:25:08.110489+0000","last_peered":"2024-04-15T15:25:08.110489+0000","last_clean":"2024-04-15T15:25:08.110489+0000","last_became_active":"2024-04-15T15:25:06.925940+0000","last_became_peered":"2024-04-15T15:25:06.925940+0000","last_unstale":"2024-04-15T15:25:08.110489+0000","last_undegraded":"2024-04-15T15:25:08.110489+0000","last_fullsized":"2024-04-15T15:25:08.110489+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:59:56.453667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110328+0000","last_change":"2024-04-15T15:25:06.929161+0000","last_active":"2024-04-15T15:25:08.110328+0000","last_peered":"2024-04-15T15:25:08.110328+0000","last_clean":"2024-04-15T15:25:08.110328+0000","last_became_active":"2024-04-15T15:25:06.927857+0000","last_became_peered":"2024-04-15T15:25:06.927857+0000","last_unstale":"2024-04-15T15:25:08.110328+0000","last_undegraded":"2024-04-15T15:25:08.110328+0000","last_fullsized":"2024-04-15T15:25:08.110328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:18:22.995620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938055+0000","last_change":"2024-04-15T15:25:06.925831+0000","last_active":"2024-04-15T15:25:07.938055+0000","last_peered":"2024-04-15T15:25:07.938055+0000","last_clean":"2024-04-15T15:25:07.938055+0000","last_became_active":"2024-04-15T15:25:06.925442+0000","last_became_peered":"2024-04-15T15:25:06.925442+0000","last_unstale":"2024-04-15T15:25:07.938055+0000","last_undegraded":"2024-04-15T15:25:07.938055+0000","last_fullsized":"2024-04-15T15:25:07.938055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:54.777764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937127+0000","last_change":"2024-04-15T15:25:06.921942+0000","last_active":"2024-04-15T15:25:07.937127+0000","last_peered":"2024-04-15T15:25:07.937127+0000","last_clean":"2024-04-15T15:25:07.937127+0000","last_became_active":"2024-04-15T15:25:06.921618+0000","last_became_peered":"2024-04-15T15:25:06.921618+0000","last_unstale":"2024-04-15T15:25:07.937127+0000","last_undegraded":"2024-04-15T15:25:07.937127+0000","last_fullsized":"2024-04-15T15:25:07.937127+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:44:23.747614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933905+0000","last_change":"2024-04-15T15:25:06.933237+0000","last_active":"2024-04-15T15:25:07.933905+0000","last_peered":"2024-04-15T15:25:07.933905+0000","last_clean":"2024-04-15T15:25:07.933905+0000","last_became_active":"2024-04-15T15:25:06.932041+0000","last_became_peered":"2024-04-15T15:25:06.932041+0000","last_unstale":"2024-04-15T15:25:07.933905+0000","last_undegraded":"2024-04-15T15:25:07.933905+0000","last_fullsized":"2024-04-15T15:25:07.933905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:23:14.407218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109849+0000","last_change":"2024-04-15T15:25:06.939659+0000","last_active":"2024-04-15T15:25:08.109849+0000","last_peered":"2024-04-15T15:25:08.109849+0000","last_clean":"2024-04-15T15:25:08.109849+0000","last_became_active":"2024-04-15T15:25:06.939087+0000","last_became_peered":"2024-04-15T15:25:06.939087+0000","last_unstale":"2024-04-15T15:25:08.109849+0000","last_undegraded":"2024-04-15T15:25:08.109849+0000","last_fullsized":"2024-04-15T15:25:08.109849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:43:24.414025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933597+0000","last_change":"2024-04-15T15:25:06.931286+0000","last_active":"2024-04-15T15:25:07.933597+0000","last_peered":"2024-04-15T15:25:07.933597+0000","last_clean":"2024-04-15T15:25:07.933597+0000","last_became_active":"2024-04-15T15:25:06.929550+0000","last_became_peered":"2024-04-15T15:25:06.929550+0000","last_unstale":"2024-04-15T15:25:07.933597+0000","last_undegraded":"2024-04-15T15:25:07.933597+0000","last_fullsized":"2024-04-15T15:25:07.933597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:46:23.614593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.940241+0000","last_change":"2024-04-15T15:25:06.913055+0000","last_active":"2024-04-15T15:25:07.940241+0000","last_peered":"2024-04-15T15:25:07.940241+0000","last_clean":"2024-04-15T15:25:07.940241+0000","last_became_active":"2024-04-15T15:25:06.912251+0000","last_became_peered":"2024-04-15T15:25:06.912251+0000","last_unstale":"2024-04-15T15:25:07.940241+0000","last_undegraded":"2024-04-15T15:25:07.940241+0000","last_fullsized":"2024-04-15T15:25:07.940241+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:51.313422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109949+0000","last_change":"2024-04-15T15:25:06.930048+0000","last_active":"2024-04-15T15:25:08.109949+0000","last_peered":"2024-04-15T15:25:08.109949+0000","last_clean":"2024-04-15T15:25:08.109949+0000","last_became_active":"2024-04-15T15:25:06.929779+0000","last_became_peered":"2024-04-15T15:25:06.929779+0000","last_unstale":"2024-04-15T15:25:08.109949+0000","last_undegraded":"2024-04-15T15:25:08.109949+0000","last_fullsized":"2024-04-15T15:25:08.109949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:52:38.724258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938063+0000","last_change":"2024-04-15T15:25:06.938089+0000","last_active":"2024-04-15T15:25:07.938063+0000","last_peered":"2024-04-15T15:25:07.938063+0000","last_clean":"2024-04-15T15:25:07.938063+0000","last_became_active":"2024-04-15T15:25:06.937742+0000","last_became_peered":"2024-04-15T15:25:06.937742+0000","last_unstale":"2024-04-15T15:25:07.938063+0000","last_undegraded":"2024-04-15T15:25:07.938063+0000","last_fullsized":"2024-04-15T15:25:07.938063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:52:48.345879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111859+0000","last_change":"2024-04-15T15:25:06.927076+0000","last_active":"2024-04-15T15:25:08.111859+0000","last_peered":"2024-04-15T15:25:08.111859+0000","last_clean":"2024-04-15T15:25:08.111859+0000","last_became_active":"2024-04-15T15:25:06.925688+0000","last_became_peered":"2024-04-15T15:25:06.925688+0000","last_unstale":"2024-04-15T15:25:08.111859+0000","last_undegraded":"2024-04-15T15:25:08.111859+0000","last_fullsized":"2024-04-15T15:25:08.111859+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:59:42.498586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937804+0000","last_change":"2024-04-15T15:25:06.937894+0000","last_active":"2024-04-15T15:25:07.937804+0000","last_peered":"2024-04-15T15:25:07.937804+0000","last_clean":"2024-04-15T15:25:07.937804+0000","last_became_active":"2024-04-15T15:25:06.937032+0000","last_became_peered":"2024-04-15T15:25:06.937032+0000","last_unstale":"2024-04-15T15:25:07.937804+0000","last_undegraded":"2024-04-15T15:25:07.937804+0000","last_fullsized":"2024-04-15T15:25:07.937804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:40.737655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938598+0000","last_change":"2024-04-15T15:25:06.930958+0000","last_active":"2024-04-15T15:25:07.938598+0000","last_peered":"2024-04-15T15:25:07.938598+0000","last_clean":"2024-04-15T15:25:07.938598+0000","last_became_active":"2024-04-15T15:25:06.930682+0000","last_became_peered":"2024-04-15T15:25:06.930682+0000","last_unstale":"2024-04-15T15:25:07.938598+0000","last_undegraded":"2024-04-15T15:25:07.938598+0000","last_fullsized":"2024-04-15T15:25:07.938598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:52:28.384976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.112309+0000","last_change":"2024-04-15T15:25:05.899821+0000","last_active":"2024-04-15T15:25:08.112309+0000","last_peered":"2024-04-15T15:25:08.112309+0000","last_clean":"2024-04-15T15:25:08.112309+0000","last_became_active":"2024-04-15T15:25:05.899522+0000","last_became_peered":"2024-04-15T15:25:05.899522+0000","last_unstale":"2024-04-15T15:25:08.112309+0000","last_undegraded":"2024-04-15T15:25:08.112309+0000","last_fullsized":"2024-04-15T15:25:08.112309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:15:40.915748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.112298+0000","last_change":"2024-04-15T15:25:06.932401+0000","last_active":"2024-04-15T15:25:08.112298+0000","last_peered":"2024-04-15T15:25:08.112298+0000","last_clean":"2024-04-15T15:25:08.112298+0000","last_became_active":"2024-04-15T15:25:06.932037+0000","last_became_peered":"2024-04-15T15:25:06.932037+0000","last_unstale":"2024-04-15T15:25:08.112298+0000","last_undegraded":"2024-04-15T15:25:08.112298+0000","last_fullsized":"2024-04-15T15:25:08.112298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:54.777764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938686+0000","last_change":"2024-04-15T15:25:05.898287+0000","last_active":"2024-04-15T15:25:07.938686+0000","last_peered":"2024-04-15T15:25:07.938686+0000","last_clean":"2024-04-15T15:25:07.938686+0000","last_became_active":"2024-04-15T15:25:05.897977+0000","last_became_peered":"2024-04-15T15:25:05.897977+0000","last_unstale":"2024-04-15T15:25:07.938686+0000","last_undegraded":"2024-04-15T15:25:07.938686+0000","last_fullsized":"2024-04-15T15:25:07.938686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:38:37.393168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110355+0000","last_change":"2024-04-15T15:25:06.931416+0000","last_active":"2024-04-15T15:25:08.110355+0000","last_peered":"2024-04-15T15:25:08.110355+0000","last_clean":"2024-04-15T15:25:08.110355+0000","last_became_active":"2024-04-15T15:25:06.930960+0000","last_became_peered":"2024-04-15T15:25:06.930960+0000","last_unstale":"2024-04-15T15:25:08.110355+0000","last_undegraded":"2024-04-15T15:25:08.110355+0000","last_fullsized":"2024-04-15T15:25:08.110355+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:52:28.384976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109294+0000","last_change":"2024-04-15T15:25:05.891999+0000","last_active":"2024-04-15T15:25:08.109294+0000","last_peered":"2024-04-15T15:25:08.109294+0000","last_clean":"2024-04-15T15:25:08.109294+0000","last_became_active":"2024-04-15T15:25:05.891707+0000","last_became_peered":"2024-04-15T15:25:05.891707+0000","last_unstale":"2024-04-15T15:25:08.109294+0000","last_undegraded":"2024-04-15T15:25:08.109294+0000","last_fullsized":"2024-04-15T15:25:08.109294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:52:37.718604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933622+0000","last_change":"2024-04-15T15:25:06.929891+0000","last_active":"2024-04-15T15:25:07.933622+0000","last_peered":"2024-04-15T15:25:07.933622+0000","last_clean":"2024-04-15T15:25:07.933622+0000","last_became_active":"2024-04-15T15:25:06.929250+0000","last_became_peered":"2024-04-15T15:25:06.929250+0000","last_unstale":"2024-04-15T15:25:07.933622+0000","last_undegraded":"2024-04-15T15:25:07.933622+0000","last_fullsized":"2024-04-15T15:25:07.933622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:24:10.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.945970+0000","last_change":"2024-04-15T15:25:05.898445+0000","last_active":"2024-04-15T15:25:06.945970+0000","last_peered":"2024-04-15T15:25:06.945970+0000","last_clean":"2024-04-15T15:25:06.945970+0000","last_became_active":"2024-04-15T15:25:05.898177+0000","last_became_peered":"2024-04-15T15:25:05.898177+0000","last_unstale":"2024-04-15T15:25:06.945970+0000","last_undegraded":"2024-04-15T15:25:06.945970+0000","last_fullsized":"2024-04-15T15:25:06.945970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:05.833643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938484+0000","last_change":"2024-04-15T15:25:06.937264+0000","last_active":"2024-04-15T15:25:07.938484+0000","last_peered":"2024-04-15T15:25:07.938484+0000","last_clean":"2024-04-15T15:25:07.938484+0000","last_became_active":"2024-04-15T15:25:06.936964+0000","last_became_peered":"2024-04-15T15:25:06.936964+0000","last_unstale":"2024-04-15T15:25:07.938484+0000","last_undegraded":"2024-04-15T15:25:07.938484+0000","last_fullsized":"2024-04-15T15:25:07.938484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:24:18.031910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110236+0000","last_change":"2024-04-15T15:25:05.900911+0000","last_active":"2024-04-15T15:25:08.110236+0000","last_peered":"2024-04-15T15:25:08.110236+0000","last_clean":"2024-04-15T15:25:08.110236+0000","last_became_active":"2024-04-15T15:25:05.900554+0000","last_became_peered":"2024-04-15T15:25:05.900554+0000","last_unstale":"2024-04-15T15:25:08.110236+0000","last_undegraded":"2024-04-15T15:25:08.110236+0000","last_fullsized":"2024-04-15T15:25:08.110236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:48:48.390397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939566+0000","last_change":"2024-04-15T15:25:06.915709+0000","last_active":"2024-04-15T15:25:07.939566+0000","last_peered":"2024-04-15T15:25:07.939566+0000","last_clean":"2024-04-15T15:25:07.939566+0000","last_became_active":"2024-04-15T15:25:06.914951+0000","last_became_peered":"2024-04-15T15:25:06.914951+0000","last_unstale":"2024-04-15T15:25:07.939566+0000","last_undegraded":"2024-04-15T15:25:07.939566+0000","last_fullsized":"2024-04-15T15:25:07.939566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:40.737655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110420+0000","last_change":"2024-04-15T15:25:05.901837+0000","last_active":"2024-04-15T15:25:08.110420+0000","last_peered":"2024-04-15T15:25:08.110420+0000","last_clean":"2024-04-15T15:25:08.110420+0000","last_became_active":"2024-04-15T15:25:05.901534+0000","last_became_peered":"2024-04-15T15:25:05.901534+0000","last_unstale":"2024-04-15T15:25:08.110420+0000","last_undegraded":"2024-04-15T15:25:08.110420+0000","last_fullsized":"2024-04-15T15:25:08.110420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:01:55.640226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109430+0000","last_change":"2024-04-15T15:25:06.929490+0000","last_active":"2024-04-15T15:25:08.109430+0000","last_peered":"2024-04-15T15:25:08.109430+0000","last_clean":"2024-04-15T15:25:08.109430+0000","last_became_active":"2024-04-15T15:25:06.929093+0000","last_became_peered":"2024-04-15T15:25:06.929093+0000","last_unstale":"2024-04-15T15:25:08.109430+0000","last_undegraded":"2024-04-15T15:25:08.109430+0000","last_fullsized":"2024-04-15T15:25:08.109430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:25:18.723432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.108464+0000","last_change":"2024-04-15T15:25:05.895397+0000","last_active":"2024-04-15T15:25:08.108464+0000","last_peered":"2024-04-15T15:25:08.108464+0000","last_clean":"2024-04-15T15:25:08.108464+0000","last_became_active":"2024-04-15T15:25:05.894274+0000","last_became_peered":"2024-04-15T15:25:05.894274+0000","last_unstale":"2024-04-15T15:25:08.108464+0000","last_undegraded":"2024-04-15T15:25:08.108464+0000","last_fullsized":"2024-04-15T15:25:08.108464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:04:54.969524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.932843+0000","last_change":"2024-04-15T15:25:06.921169+0000","last_active":"2024-04-15T15:25:07.932843+0000","last_peered":"2024-04-15T15:25:07.932843+0000","last_clean":"2024-04-15T15:25:07.932843+0000","last_became_active":"2024-04-15T15:25:06.920875+0000","last_became_peered":"2024-04-15T15:25:06.920875+0000","last_unstale":"2024-04-15T15:25:07.932843+0000","last_undegraded":"2024-04-15T15:25:07.932843+0000","last_fullsized":"2024-04-15T15:25:07.932843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:29:51.053513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109341+0000","last_change":"2024-04-15T15:25:05.896020+0000","last_active":"2024-04-15T15:25:08.109341+0000","last_peered":"2024-04-15T15:25:08.109341+0000","last_clean":"2024-04-15T15:25:08.109341+0000","last_became_active":"2024-04-15T15:25:05.895759+0000","last_became_peered":"2024-04-15T15:25:05.895759+0000","last_unstale":"2024-04-15T15:25:08.109341+0000","last_undegraded":"2024-04-15T15:25:08.109341+0000","last_fullsized":"2024-04-15T15:25:08.109341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:31.798053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939943+0000","last_change":"2024-04-15T15:25:06.928693+0000","last_active":"2024-04-15T15:25:07.939943+0000","last_peered":"2024-04-15T15:25:07.939943+0000","last_clean":"2024-04-15T15:25:07.939943+0000","last_became_active":"2024-04-15T15:25:06.928375+0000","last_became_peered":"2024-04-15T15:25:06.928375+0000","last_unstale":"2024-04-15T15:25:07.939943+0000","last_undegraded":"2024-04-15T15:25:07.939943+0000","last_fullsized":"2024-04-15T15:25:07.939943+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:52:48.345879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109811+0000","last_change":"2024-04-15T15:25:05.897944+0000","last_active":"2024-04-15T15:25:08.109811+0000","last_peered":"2024-04-15T15:25:08.109811+0000","last_clean":"2024-04-15T15:25:08.109811+0000","last_became_active":"2024-04-15T15:25:05.897634+0000","last_became_peered":"2024-04-15T15:25:05.897634+0000","last_unstale":"2024-04-15T15:25:08.109811+0000","last_undegraded":"2024-04-15T15:25:08.109811+0000","last_fullsized":"2024-04-15T15:25:08.109811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:53.772110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109669+0000","last_change":"2024-04-15T15:25:06.938843+0000","last_active":"2024-04-15T15:25:08.109669+0000","last_peered":"2024-04-15T15:25:08.109669+0000","last_clean":"2024-04-15T15:25:08.109669+0000","last_became_active":"2024-04-15T15:25:06.938560+0000","last_became_peered":"2024-04-15T15:25:06.938560+0000","last_unstale":"2024-04-15T15:25:08.109669+0000","last_undegraded":"2024-04-15T15:25:08.109669+0000","last_fullsized":"2024-04-15T15:25:08.109669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:52:48.345879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938901+0000","last_change":"2024-04-15T15:25:05.899918+0000","last_active":"2024-04-15T15:25:07.938901+0000","last_peered":"2024-04-15T15:25:07.938901+0000","last_clean":"2024-04-15T15:25:07.938901+0000","last_became_active":"2024-04-15T15:25:05.899315+0000","last_became_peered":"2024-04-15T15:25:05.899315+0000","last_unstale":"2024-04-15T15:25:07.938901+0000","last_undegraded":"2024-04-15T15:25:07.938901+0000","last_fullsized":"2024-04-15T15:25:07.938901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:57:52.551727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109250+0000","last_change":"2024-04-15T15:25:06.939452+0000","last_active":"2024-04-15T15:25:08.109250+0000","last_peered":"2024-04-15T15:25:08.109250+0000","last_clean":"2024-04-15T15:25:08.109250+0000","last_became_active":"2024-04-15T15:25:06.939106+0000","last_became_peered":"2024-04-15T15:25:06.939106+0000","last_unstale":"2024-04-15T15:25:08.109250+0000","last_undegraded":"2024-04-15T15:25:08.109250+0000","last_fullsized":"2024-04-15T15:25:08.109250+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:40.737655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939821+0000","last_change":"2024-04-15T15:25:05.897110+0000","last_active":"2024-04-15T15:25:07.939821+0000","last_peered":"2024-04-15T15:25:07.939821+0000","last_clean":"2024-04-15T15:25:07.939821+0000","last_became_active":"2024-04-15T15:25:05.896827+0000","last_became_peered":"2024-04-15T15:25:05.896827+0000","last_unstale":"2024-04-15T15:25:07.939821+0000","last_undegraded":"2024-04-15T15:25:07.939821+0000","last_fullsized":"2024-04-15T15:25:07.939821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:25:17.717778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939432+0000","last_change":"2024-04-15T15:25:06.935416+0000","last_active":"2024-04-15T15:25:07.939432+0000","last_peered":"2024-04-15T15:25:07.939432+0000","last_clean":"2024-04-15T15:25:07.939432+0000","last_became_active":"2024-04-15T15:25:06.934906+0000","last_became_peered":"2024-04-15T15:25:06.934906+0000","last_unstale":"2024-04-15T15:25:07.939432+0000","last_undegraded":"2024-04-15T15:25:07.939432+0000","last_fullsized":"2024-04-15T15:25:07.939432+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:12.196977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109263+0000","last_change":"2024-04-15T15:25:05.895267+0000","last_active":"2024-04-15T15:25:08.109263+0000","last_peered":"2024-04-15T15:25:08.109263+0000","last_clean":"2024-04-15T15:25:08.109263+0000","last_became_active":"2024-04-15T15:25:05.894993+0000","last_became_peered":"2024-04-15T15:25:05.894993+0000","last_unstale":"2024-04-15T15:25:08.109263+0000","last_undegraded":"2024-04-15T15:25:08.109263+0000","last_fullsized":"2024-04-15T15:25:08.109263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:25:17.717778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.932846+0000","last_change":"2024-04-15T15:25:06.922802+0000","last_active":"2024-04-15T15:25:07.932846+0000","last_peered":"2024-04-15T15:25:07.932846+0000","last_clean":"2024-04-15T15:25:07.932846+0000","last_became_active":"2024-04-15T15:25:06.922488+0000","last_became_peered":"2024-04-15T15:25:06.922488+0000","last_unstale":"2024-04-15T15:25:07.932846+0000","last_undegraded":"2024-04-15T15:25:07.932846+0000","last_fullsized":"2024-04-15T15:25:07.932846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:42.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110532+0000","last_change":"2024-04-15T15:25:05.899710+0000","last_active":"2024-04-15T15:25:08.110532+0000","last_peered":"2024-04-15T15:25:08.110532+0000","last_clean":"2024-04-15T15:25:08.110532+0000","last_became_active":"2024-04-15T15:25:05.899414+0000","last_became_peered":"2024-04-15T15:25:05.899414+0000","last_unstale":"2024-04-15T15:25:08.110532+0000","last_undegraded":"2024-04-15T15:25:08.110532+0000","last_fullsized":"2024-04-15T15:25:08.110532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:27:53.296965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109329+0000","last_change":"2024-04-15T15:25:06.939727+0000","last_active":"2024-04-15T15:25:08.109329+0000","last_peered":"2024-04-15T15:25:08.109329+0000","last_clean":"2024-04-15T15:25:08.109329+0000","last_became_active":"2024-04-15T15:25:06.936112+0000","last_became_peered":"2024-04-15T15:25:06.936112+0000","last_unstale":"2024-04-15T15:25:08.109329+0000","last_undegraded":"2024-04-15T15:25:08.109329+0000","last_fullsized":"2024-04-15T15:25:08.109329+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:52:59.487388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.932868+0000","last_change":"2024-04-15T15:25:05.892271+0000","last_active":"2024-04-15T15:25:07.932868+0000","last_peered":"2024-04-15T15:25:07.932868+0000","last_clean":"2024-04-15T15:25:07.932868+0000","last_became_active":"2024-04-15T15:25:05.890984+0000","last_became_peered":"2024-04-15T15:25:05.890984+0000","last_unstale":"2024-04-15T15:25:07.932868+0000","last_undegraded":"2024-04-15T15:25:07.932868+0000","last_fullsized":"2024-04-15T15:25:07.932868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:57:52.551727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937811+0000","last_change":"2024-04-15T15:25:06.936599+0000","last_active":"2024-04-15T15:25:07.937811+0000","last_peered":"2024-04-15T15:25:07.937811+0000","last_clean":"2024-04-15T15:25:07.937811+0000","last_became_active":"2024-04-15T15:25:06.936222+0000","last_became_peered":"2024-04-15T15:25:06.936222+0000","last_unstale":"2024-04-15T15:25:07.937811+0000","last_undegraded":"2024-04-15T15:25:07.937811+0000","last_fullsized":"2024-04-15T15:25:07.937811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:52:59.487388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.108702+0000","last_change":"2024-04-15T15:25:05.896797+0000","last_active":"2024-04-15T15:25:08.108702+0000","last_peered":"2024-04-15T15:25:08.108702+0000","last_clean":"2024-04-15T15:25:08.108702+0000","last_became_active":"2024-04-15T15:25:05.896492+0000","last_became_peered":"2024-04-15T15:25:05.896492+0000","last_unstale":"2024-04-15T15:25:08.108702+0000","last_undegraded":"2024-04-15T15:25:08.108702+0000","last_fullsized":"2024-04-15T15:25:08.108702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:20:36.568994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111378+0000","last_change":"2024-04-15T15:25:06.929759+0000","last_active":"2024-04-15T15:25:08.111378+0000","last_peered":"2024-04-15T15:25:08.111378+0000","last_clean":"2024-04-15T15:25:08.111378+0000","last_became_active":"2024-04-15T15:25:06.929466+0000","last_became_peered":"2024-04-15T15:25:06.929466+0000","last_unstale":"2024-04-15T15:25:08.111378+0000","last_undegraded":"2024-04-15T15:25:08.111378+0000","last_fullsized":"2024-04-15T15:25:08.111378+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:20:37.574648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.107768+0000","last_change":"2024-04-15T15:25:05.890106+0000","last_active":"2024-04-15T15:25:08.107768+0000","last_peered":"2024-04-15T15:25:08.107768+0000","last_clean":"2024-04-15T15:25:08.107768+0000","last_became_active":"2024-04-15T15:25:05.889812+0000","last_became_peered":"2024-04-15T15:25:05.889812+0000","last_unstale":"2024-04-15T15:25:08.107768+0000","last_undegraded":"2024-04-15T15:25:08.107768+0000","last_fullsized":"2024-04-15T15:25:08.107768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:19:07.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109869+0000","last_change":"2024-04-15T15:25:06.928928+0000","last_active":"2024-04-15T15:25:08.109869+0000","last_peered":"2024-04-15T15:25:08.109869+0000","last_clean":"2024-04-15T15:25:08.109869+0000","last_became_active":"2024-04-15T15:25:06.923616+0000","last_became_peered":"2024-04-15T15:25:06.923616+0000","last_unstale":"2024-04-15T15:25:08.109869+0000","last_undegraded":"2024-04-15T15:25:08.109869+0000","last_fullsized":"2024-04-15T15:25:08.109869+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:54.777764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.934054+0000","last_change":"2024-04-15T15:25:05.885558+0000","last_active":"2024-04-15T15:25:07.934054+0000","last_peered":"2024-04-15T15:25:07.934054+0000","last_clean":"2024-04-15T15:25:07.934054+0000","last_became_active":"2024-04-15T15:25:05.884844+0000","last_became_peered":"2024-04-15T15:25:05.884844+0000","last_unstale":"2024-04-15T15:25:07.934054+0000","last_undegraded":"2024-04-15T15:25:07.934054+0000","last_fullsized":"2024-04-15T15:25:07.934054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:20:36.568994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939931+0000","last_change":"2024-04-15T15:25:06.924292+0000","last_active":"2024-04-15T15:25:07.939931+0000","last_peered":"2024-04-15T15:25:07.939931+0000","last_clean":"2024-04-15T15:25:07.939931+0000","last_became_active":"2024-04-15T15:25:06.923978+0000","last_became_peered":"2024-04-15T15:25:06.923978+0000","last_unstale":"2024-04-15T15:25:07.939931+0000","last_undegraded":"2024-04-15T15:25:07.939931+0000","last_fullsized":"2024-04-15T15:25:07.939931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:52:59.487388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110140+0000","last_change":"2024-04-15T15:25:05.902120+0000","last_active":"2024-04-15T15:25:08.110140+0000","last_peered":"2024-04-15T15:25:08.110140+0000","last_clean":"2024-04-15T15:25:08.110140+0000","last_became_active":"2024-04-15T15:25:05.901793+0000","last_became_peered":"2024-04-15T15:25:05.901793+0000","last_unstale":"2024-04-15T15:25:08.110140+0000","last_undegraded":"2024-04-15T15:25:08.110140+0000","last_fullsized":"2024-04-15T15:25:08.110140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:46:31.178019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933408+0000","last_change":"2024-04-15T15:25:06.921502+0000","last_active":"2024-04-15T15:25:07.933408+0000","last_peered":"2024-04-15T15:25:07.933408+0000","last_clean":"2024-04-15T15:25:07.933408+0000","last_became_active":"2024-04-15T15:25:06.921155+0000","last_became_peered":"2024-04-15T15:25:06.921155+0000","last_unstale":"2024-04-15T15:25:07.933408+0000","last_undegraded":"2024-04-15T15:25:07.933408+0000","last_fullsized":"2024-04-15T15:25:07.933408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:29:56.797123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110483+0000","last_change":"2024-04-15T15:25:05.903279+0000","last_active":"2024-04-15T15:25:08.110483+0000","last_peered":"2024-04-15T15:25:08.110483+0000","last_clean":"2024-04-15T15:25:08.110483+0000","last_became_active":"2024-04-15T15:25:05.902981+0000","last_became_peered":"2024-04-15T15:25:05.902981+0000","last_unstale":"2024-04-15T15:25:08.110483+0000","last_undegraded":"2024-04-15T15:25:08.110483+0000","last_fullsized":"2024-04-15T15:25:08.110483+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:28.015619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.934487+0000","last_change":"2024-04-15T15:25:06.926613+0000","last_active":"2024-04-15T15:25:06.934487+0000","last_peered":"2024-04-15T15:25:06.934487+0000","last_clean":"2024-04-15T15:25:06.934487+0000","last_became_active":"2024-04-15T15:25:06.926347+0000","last_became_peered":"2024-04-15T15:25:06.926347+0000","last_unstale":"2024-04-15T15:25:06.934487+0000","last_undegraded":"2024-04-15T15:25:06.934487+0000","last_fullsized":"2024-04-15T15:25:06.934487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:42:22.989173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110635+0000","last_change":"2024-04-15T15:25:05.900982+0000","last_active":"2024-04-15T15:25:08.110635+0000","last_peered":"2024-04-15T15:25:08.110635+0000","last_clean":"2024-04-15T15:25:08.110635+0000","last_became_active":"2024-04-15T15:25:05.900568+0000","last_became_peered":"2024-04-15T15:25:05.900568+0000","last_unstale":"2024-04-15T15:25:08.110635+0000","last_undegraded":"2024-04-15T15:25:08.110635+0000","last_fullsized":"2024-04-15T15:25:08.110635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:03:56.673616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.931264+0000","last_change":"2024-04-15T15:25:06.925851+0000","last_active":"2024-04-15T15:25:06.931264+0000","last_peered":"2024-04-15T15:25:06.931264+0000","last_clean":"2024-04-15T15:25:06.931264+0000","last_became_active":"2024-04-15T15:25:06.923472+0000","last_became_peered":"2024-04-15T15:25:06.923472+0000","last_unstale":"2024-04-15T15:25:06.931264+0000","last_undegraded":"2024-04-15T15:25:06.931264+0000","last_fullsized":"2024-04-15T15:25:06.931264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:51:37.853716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938257+0000","last_change":"2024-04-15T15:25:05.891567+0000","last_active":"2024-04-15T15:25:07.938257+0000","last_peered":"2024-04-15T15:25:07.938257+0000","last_clean":"2024-04-15T15:25:07.938257+0000","last_became_active":"2024-04-15T15:25:05.890204+0000","last_became_peered":"2024-04-15T15:25:05.890204+0000","last_unstale":"2024-04-15T15:25:07.938257+0000","last_undegraded":"2024-04-15T15:25:07.938257+0000","last_fullsized":"2024-04-15T15:25:07.938257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:15:40.915748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.921767+0000","last_change":"2024-04-15T15:25:06.909790+0000","last_active":"2024-04-15T15:25:06.921767+0000","last_peered":"2024-04-15T15:25:06.921767+0000","last_clean":"2024-04-15T15:25:06.921767+0000","last_became_active":"2024-04-15T15:25:06.908954+0000","last_became_peered":"2024-04-15T15:25:06.908954+0000","last_unstale":"2024-04-15T15:25:06.921767+0000","last_undegraded":"2024-04-15T15:25:06.921767+0000","last_fullsized":"2024-04-15T15:25:06.921767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:32:20.401368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111469+0000","last_change":"2024-04-15T15:25:05.896334+0000","last_active":"2024-04-15T15:25:08.111469+0000","last_peered":"2024-04-15T15:25:08.111469+0000","last_clean":"2024-04-15T15:25:08.111469+0000","last_became_active":"2024-04-15T15:25:05.896051+0000","last_became_peered":"2024-04-15T15:25:05.896051+0000","last_unstale":"2024-04-15T15:25:08.111469+0000","last_undegraded":"2024-04-15T15:25:08.111469+0000","last_fullsized":"2024-04-15T15:25:08.111469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:28.015619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.921964+0000","last_change":"2024-04-15T15:25:06.911563+0000","last_active":"2024-04-15T15:25:06.921964+0000","last_peered":"2024-04-15T15:25:06.921964+0000","last_clean":"2024-04-15T15:25:06.921964+0000","last_became_active":"2024-04-15T15:25:06.910792+0000","last_became_peered":"2024-04-15T15:25:06.910792+0000","last_unstale":"2024-04-15T15:25:06.921964+0000","last_undegraded":"2024-04-15T15:25:06.921964+0000","last_fullsized":"2024-04-15T15:25:06.921964+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:04:51.287497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109440+0000","last_change":"2024-04-15T15:25:05.903187+0000","last_active":"2024-04-15T15:25:08.109440+0000","last_peered":"2024-04-15T15:25:08.109440+0000","last_clean":"2024-04-15T15:25:08.109440+0000","last_became_active":"2024-04-15T15:25:05.900973+0000","last_became_peered":"2024-04-15T15:25:05.900973+0000","last_unstale":"2024-04-15T15:25:08.109440+0000","last_undegraded":"2024-04-15T15:25:08.109440+0000","last_fullsized":"2024-04-15T15:25:08.109440+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:30:40.996483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938083+0000","last_change":"2024-04-15T15:25:06.931216+0000","last_active":"2024-04-15T15:25:07.938083+0000","last_peered":"2024-04-15T15:25:07.938083+0000","last_clean":"2024-04-15T15:25:07.938083+0000","last_became_active":"2024-04-15T15:25:06.930712+0000","last_became_peered":"2024-04-15T15:25:06.930712+0000","last_unstale":"2024-04-15T15:25:07.938083+0000","last_undegraded":"2024-04-15T15:25:07.938083+0000","last_fullsized":"2024-04-15T15:25:07.938083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:03.905713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937375+0000","last_change":"2024-04-15T15:25:05.900424+0000","last_active":"2024-04-15T15:25:07.937375+0000","last_peered":"2024-04-15T15:25:07.937375+0000","last_clean":"2024-04-15T15:25:07.937375+0000","last_became_active":"2024-04-15T15:25:05.900139+0000","last_became_peered":"2024-04-15T15:25:05.900139+0000","last_unstale":"2024-04-15T15:25:07.937375+0000","last_undegraded":"2024-04-15T15:25:07.937375+0000","last_fullsized":"2024-04-15T15:25:07.937375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:53.772110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.918132+0000","last_change":"2024-04-15T15:25:06.909587+0000","last_active":"2024-04-15T15:25:06.918132+0000","last_peered":"2024-04-15T15:25:06.918132+0000","last_clean":"2024-04-15T15:25:06.918132+0000","last_became_active":"2024-04-15T15:25:06.908974+0000","last_became_peered":"2024-04-15T15:25:06.908974+0000","last_unstale":"2024-04-15T15:25:06.918132+0000","last_undegraded":"2024-04-15T15:25:06.918132+0000","last_fullsized":"2024-04-15T15:25:06.918132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:53:31.878708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.934063+0000","last_change":"2024-04-15T15:25:05.880651+0000","last_active":"2024-04-15T15:25:07.934063+0000","last_peered":"2024-04-15T15:25:07.934063+0000","last_clean":"2024-04-15T15:25:07.934063+0000","last_became_active":"2024-04-15T15:25:05.880298+0000","last_became_peered":"2024-04-15T15:25:05.880298+0000","last_unstale":"2024-04-15T15:25:07.934063+0000","last_undegraded":"2024-04-15T15:25:07.934063+0000","last_fullsized":"2024-04-15T15:25:07.934063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:19:07.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.940005+0000","last_change":"2024-04-15T15:25:06.922016+0000","last_active":"2024-04-15T15:25:07.940005+0000","last_peered":"2024-04-15T15:25:07.940005+0000","last_clean":"2024-04-15T15:25:07.940005+0000","last_became_active":"2024-04-15T15:25:06.921260+0000","last_became_peered":"2024-04-15T15:25:06.921260+0000","last_unstale":"2024-04-15T15:25:07.940005+0000","last_undegraded":"2024-04-15T15:25:07.940005+0000","last_fullsized":"2024-04-15T15:25:07.940005+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:03.905713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.942056+0000","last_change":"2024-04-15T15:25:05.890841+0000","last_active":"2024-04-15T15:25:07.942056+0000","last_peered":"2024-04-15T15:25:07.942056+0000","last_clean":"2024-04-15T15:25:07.942056+0000","last_became_active":"2024-04-15T15:25:05.889042+0000","last_became_peered":"2024-04-15T15:25:05.889042+0000","last_unstale":"2024-04-15T15:25:07.942056+0000","last_undegraded":"2024-04-15T15:25:07.942056+0000","last_fullsized":"2024-04-15T15:25:07.942056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:42.892474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938369+0000","last_change":"2024-04-15T15:25:06.932945+0000","last_active":"2024-04-15T15:25:07.938369+0000","last_peered":"2024-04-15T15:25:07.938369+0000","last_clean":"2024-04-15T15:25:07.938369+0000","last_became_active":"2024-04-15T15:25:06.931880+0000","last_became_peered":"2024-04-15T15:25:06.931880+0000","last_unstale":"2024-04-15T15:25:07.938369+0000","last_undegraded":"2024-04-15T15:25:07.938369+0000","last_fullsized":"2024-04-15T15:25:07.938369+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:18:22.995620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933407+0000","last_change":"2024-04-15T15:25:05.894631+0000","last_active":"2024-04-15T15:25:07.933407+0000","last_peered":"2024-04-15T15:25:07.933407+0000","last_clean":"2024-04-15T15:25:07.933407+0000","last_became_active":"2024-04-15T15:25:05.894303+0000","last_became_peered":"2024-04-15T15:25:05.894303+0000","last_unstale":"2024-04-15T15:25:07.933407+0000","last_undegraded":"2024-04-15T15:25:07.933407+0000","last_fullsized":"2024-04-15T15:25:07.933407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:04:54.969524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111501+0000","last_change":"2024-04-15T15:25:06.933286+0000","last_active":"2024-04-15T15:25:08.111501+0000","last_peered":"2024-04-15T15:25:08.111501+0000","last_clean":"2024-04-15T15:25:08.111501+0000","last_became_active":"2024-04-15T15:25:06.932559+0000","last_became_peered":"2024-04-15T15:25:06.932559+0000","last_unstale":"2024-04-15T15:25:08.111501+0000","last_undegraded":"2024-04-15T15:25:08.111501+0000","last_fullsized":"2024-04-15T15:25:08.111501+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:19:08.636594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938445+0000","last_change":"2024-04-15T15:25:05.892283+0000","last_active":"2024-04-15T15:25:07.938445+0000","last_peered":"2024-04-15T15:25:07.938445+0000","last_clean":"2024-04-15T15:25:07.938445+0000","last_became_active":"2024-04-15T15:25:05.891991+0000","last_became_peered":"2024-04-15T15:25:05.891991+0000","last_unstale":"2024-04-15T15:25:07.938445+0000","last_undegraded":"2024-04-15T15:25:07.938445+0000","last_fullsized":"2024-04-15T15:25:07.938445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:18:18.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938360+0000","last_change":"2024-04-15T15:25:06.932407+0000","last_active":"2024-04-15T15:25:07.938360+0000","last_peered":"2024-04-15T15:25:07.938360+0000","last_clean":"2024-04-15T15:25:07.938360+0000","last_became_active":"2024-04-15T15:25:06.931130+0000","last_became_peered":"2024-04-15T15:25:06.931130+0000","last_unstale":"2024-04-15T15:25:07.938360+0000","last_undegraded":"2024-04-15T15:25:07.938360+0000","last_fullsized":"2024-04-15T15:25:07.938360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:52:38.724258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.932984+0000","last_change":"2024-04-15T15:25:05.880000+0000","last_active":"2024-04-15T15:25:07.932984+0000","last_peered":"2024-04-15T15:25:07.932984+0000","last_clean":"2024-04-15T15:25:07.932984+0000","last_became_active":"2024-04-15T15:25:05.879548+0000","last_became_peered":"2024-04-15T15:25:05.879548+0000","last_unstale":"2024-04-15T15:25:07.932984+0000","last_undegraded":"2024-04-15T15:25:07.932984+0000","last_fullsized":"2024-04-15T15:25:07.932984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:42.892474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.107687+0000","last_change":"2024-04-15T15:25:06.927133+0000","last_active":"2024-04-15T15:25:08.107687+0000","last_peered":"2024-04-15T15:25:08.107687+0000","last_clean":"2024-04-15T15:25:08.107687+0000","last_became_active":"2024-04-15T15:25:06.926840+0000","last_became_peered":"2024-04-15T15:25:06.926840+0000","last_unstale":"2024-04-15T15:25:08.107687+0000","last_undegraded":"2024-04-15T15:25:08.107687+0000","last_fullsized":"2024-04-15T15:25:08.107687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:44:23.747614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109706+0000","last_change":"2024-04-15T15:25:05.902440+0000","last_active":"2024-04-15T15:25:08.109706+0000","last_peered":"2024-04-15T15:25:08.109706+0000","last_clean":"2024-04-15T15:25:08.109706+0000","last_became_active":"2024-04-15T15:25:05.901490+0000","last_became_peered":"2024-04-15T15:25:05.901490+0000","last_unstale":"2024-04-15T15:25:08.109706+0000","last_undegraded":"2024-04-15T15:25:08.109706+0000","last_fullsized":"2024-04-15T15:25:08.109706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:10.254803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110632+0000","last_change":"2024-04-15T15:25:06.929366+0000","last_active":"2024-04-15T15:25:08.110632+0000","last_peered":"2024-04-15T15:25:08.110632+0000","last_clean":"2024-04-15T15:25:08.110632+0000","last_became_active":"2024-04-15T15:25:06.928041+0000","last_became_peered":"2024-04-15T15:25:06.928041+0000","last_unstale":"2024-04-15T15:25:08.110632+0000","last_undegraded":"2024-04-15T15:25:08.110632+0000","last_fullsized":"2024-04-15T15:25:08.110632+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:38.283098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.107835+0000","last_change":"2024-04-15T15:25:05.900976+0000","last_active":"2024-04-15T15:25:08.107835+0000","last_peered":"2024-04-15T15:25:08.107835+0000","last_clean":"2024-04-15T15:25:08.107835+0000","last_became_active":"2024-04-15T15:25:05.900780+0000","last_became_peered":"2024-04-15T15:25:05.900780+0000","last_unstale":"2024-04-15T15:25:08.107835+0000","last_undegraded":"2024-04-15T15:25:08.107835+0000","last_fullsized":"2024-04-15T15:25:08.107835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:59:41.492932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938315+0000","last_change":"2024-04-15T15:25:06.934205+0000","last_active":"2024-04-15T15:25:07.938315+0000","last_peered":"2024-04-15T15:25:07.938315+0000","last_clean":"2024-04-15T15:25:07.938315+0000","last_became_active":"2024-04-15T15:25:06.933881+0000","last_became_peered":"2024-04-15T15:25:06.933881+0000","last_unstale":"2024-04-15T15:25:07.938315+0000","last_undegraded":"2024-04-15T15:25:07.938315+0000","last_fullsized":"2024-04-15T15:25:07.938315+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:51.313422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111424+0000","last_change":"2024-04-15T15:25:05.905016+0000","last_active":"2024-04-15T15:25:08.111424+0000","last_peered":"2024-04-15T15:25:08.111424+0000","last_clean":"2024-04-15T15:25:08.111424+0000","last_became_active":"2024-04-15T15:25:05.904813+0000","last_became_peered":"2024-04-15T15:25:05.904813+0000","last_unstale":"2024-04-15T15:25:08.111424+0000","last_undegraded":"2024-04-15T15:25:08.111424+0000","last_fullsized":"2024-04-15T15:25:08.111424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:03:56.673616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938252+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:25:07.938252+0000","last_peered":"2024-04-15T15:25:07.938252+0000","last_clean":"2024-04-15T15:25:07.938252+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:25:07.938252+0000","last_undegraded":"2024-04-15T15:25:07.938252+0000","last_fullsized":"2024-04-15T15:25:07.938252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":736464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":146622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":146110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":147640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":147640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":736374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:25:11.379 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:25:11.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.632+0000 7f823be00640 1 Processor -- start 2024-04-15T15:25:11.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.632+0000 7f823be00640 1 -- start start 2024-04-15T15:25:11.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.633+0000 7f823be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f823c0cee00 0x7f823c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.633+0000 7f823be00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f823c15dbb0 con 0x7f823c15d2f0 2024-04-15T15:25:11.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.633+0000 7f823be00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f823c15d8d0 con 0x7f823c071f50 2024-04-15T15:25:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.633+0000 7f823be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f823c06f4f0 con 0x7f823c0cee00 2024-04-15T15:25:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.633+0000 7f823b400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f823c15d2f0 0x7f823c15dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:55186/0 (socket says 172.21.15.150:55186) 2024-04-15T15:25:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.633+0000 7f823b400640 1 -- 172.21.15.150:0/2142485504 learned_addr learned my addr 172.21.15.150:0/2142485504 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.633+0000 7f823aa00640 1 --2- 172.21.15.150:0/2142485504 >> v2:172.21.15.187:3300/0 conn(0x7f823c0cee00 0x7f823c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075708957 0 0) 0x7f823c15dbb0 con 0x7f823c15d2f0 2024-04-15T15:25:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8220003520 con 0x7f823c15d2f0 2024-04-15T15:25:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2938019132 0 0) 0x7f8220003520 con 0x7f823c15d2f0 2024-04-15T15:25:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 >> v2:172.21.15.187:3300/0 conn(0x7f823c0cee00 msgr2=0x7f823c06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 --2- 172.21.15.150:0/2142485504 >> v2:172.21.15.187:3300/0 conn(0x7f823c0cee00 0x7f823c06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 >> v1:172.21.15.187:6789/0 conn(0x7f823c071f50 legacy=0x7f823c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c06dde0 con 0x7f823c15d2f0 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f82340019f0 con 0x7f823c15d2f0 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 >> v1:172.21.15.150:6789/0 conn(0x7f823c15d2f0 legacy=0x7f823c15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82200039b0 0x7f8220003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8220004800 0x7f8220004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82200051c0 0x7f822000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f822000c030 con 0x7f8220004800 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f822000c1e0 con 0x7f82200039b0 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.634+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f822000c3e0 con 0x7f82200051c0 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f8232000640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8220004800 0x7f8220004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f823aa00640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82200039b0 0x7f8220003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f823b400640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82200051c0 0x7f822000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f8232000640 1 -- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82200051c0 msgr2=0x7f822000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f8232000640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82200051c0 0x7f822000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f8232000640 1 -- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82200039b0 msgr2=0x7f8220003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f8232000640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82200039b0 0x7f8220003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f8232000640 1 -- 172.21.15.150:0/2142485504 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8220003520 con 0x7f8220004800 2024-04-15T15:25:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f823b400640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82200051c0 0x7f822000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.635+0000 7f823b400640 1 -- 172.21.15.150:0/2142485504 reap_dead start 2024-04-15T15:25:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.636+0000 7f8232000640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8220004800 0x7f8220004c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8224001750 tx=0x7f8220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791a3a47ed697547 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.636+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224001370 con 0x7f8220004800 2024-04-15T15:25:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82240052d0 con 0x7f8220004800 2024-04-15T15:25:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823a000640 1 -- 172.21.15.150:0/2142485504 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224003830 con 0x7f8220004800 2024-04-15T15:25:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823be00640 1 -- 172.21.15.150:0/2142485504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8220004800 msgr2=0x7f8220004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823be00640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8220004800 0x7f8220004c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8224001750 tx=0x7f8220005830 comp rx=0 tx=0).stop 2024-04-15T15:25:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823be00640 1 -- 172.21.15.150:0/2142485504 shutdown_connections 2024-04-15T15:25:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823be00640 1 --2- 172.21.15.150:0/2142485504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8220004800 0x7f8220004c80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823be00640 1 -- 172.21.15.150:0/2142485504 >> 172.21.15.150:0/2142485504 conn(0x7f823c06b930 msgr2=0x7f823c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823be00640 1 -- 172.21.15.150:0/2142485504 shutdown_connections 2024-04-15T15:25:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.637+0000 7f823be00640 1 -- 172.21.15.150:0/2142485504 wait complete. 2024-04-15T15:25:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.639+0000 7f823be00640 1 Processor -- start 2024-04-15T15:25:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.639+0000 7f823be00640 1 -- start start 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.640+0000 7f823be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8220004800 0x7f823c14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.640+0000 7f823be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f823c14c0a0 0x7f823c141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.640+0000 7f823be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c155620 0x7f823c153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.640+0000 7f823be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f823c06e8c0 con 0x7f823c155620 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.640+0000 7f823be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f823c06e3a0 con 0x7f8220004800 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.640+0000 7f823be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f823c06ea40 con 0x7f823c14c0a0 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.640+0000 7f823b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c155620 0x7f823c153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c155620 0x7f823c153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55242/0 (socket says 172.21.15.150:55242) 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823b400640 1 -- 172.21.15.150:0/2178571092 learned_addr learned my addr 172.21.15.150:0/2178571092 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823aa00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8220004800 0x7f823c14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f8232000640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f823c14c0a0 0x7f823c141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823b400640 1 -- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f823c14c0a0 msgr2=0x7f823c141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823b400640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f823c14c0a0 0x7f823c141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823b400640 1 -- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8220004800 msgr2=0x7f823c14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823b400640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8220004800 0x7f823c14f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823b400640 1 -- 172.21.15.150:0/2178571092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c15e240 con 0x7f823c155620 2024-04-15T15:25:11.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.641+0000 7f823b400640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c155620 0x7f823c153650 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f823c161270 tx=0x7f823400b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.642+0000 7f8233400640 1 -- 172.21.15.150:0/2178571092 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823400b870 con 0x7f823c155620 2024-04-15T15:25:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.642+0000 7f8233400640 1 -- 172.21.15.150:0/2178571092 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823400ba30 con 0x7f823c155620 2024-04-15T15:25:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.642+0000 7f8233400640 1 -- 172.21.15.150:0/2178571092 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823400be20 con 0x7f823c155620 2024-04-15T15:25:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.642+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c15e520 con 0x7f823c155620 2024-04-15T15:25:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.642+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c15eae0 con 0x7f823c155620 2024-04-15T15:25:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.643+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c0755a0 con 0x7f823c155620 2024-04-15T15:25:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.644+0000 7f8233400640 1 -- 172.21.15.150:0/2178571092 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f823401b020 con 0x7f823c155620 2024-04-15T15:25:11.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.645+0000 7f8233400640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:11.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.645+0000 7f8233400640 1 -- 172.21.15.150:0/2178571092 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f823408bbc0 con 0x7f823c155620 2024-04-15T15:25:11.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.645+0000 7f823aa00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:11.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.647+0000 7f823aa00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f821806c8f0 0x7f821806edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f822c009a90 tx=0x7f822c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:11.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.650+0000 7f8233400640 1 -- 172.21.15.150:0/2178571092 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8234059f50 con 0x7f823c155620 2024-04-15T15:25:11.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.844+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f823c15dff0 con 0x7f821806c8f0 2024-04-15T15:25:11.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.867+0000 7f8233400640 1 -- 172.21.15.150:0/2178571092 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f823c15dff0 con 0x7f821806c8f0 2024-04-15T15:25:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:11.872 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:25:11.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f821806c8f0 msgr2=0x7f821806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f821806c8f0 0x7f821806edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f822c009a90 tx=0x7f822c008040 comp rx=0 tx=0).stop 2024-04-15T15:25:11.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c155620 msgr2=0x7f823c153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:11.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c155620 0x7f823c153650 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f823c161270 tx=0x7f823400b0d0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 shutdown_connections 2024-04-15T15:25:11.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c155620 0x7f823c153650 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f823c14c0a0 0x7f823c141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 --2- 172.21.15.150:0/2178571092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8220004800 0x7f823c14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:11.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 >> 172.21.15.150:0/2178571092 conn(0x7f823c06b930 msgr2=0x7f823c159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:11.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 shutdown_connections 2024-04-15T15:25:11.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:11.872+0000 7f823be00640 1 -- 172.21.15.150:0/2178571092 wait complete. 2024-04-15T15:25:11.893 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-15T15:25:11.243224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220356,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754364,"statfs":{"total":773094113280,"available":772868468736,"internally_reserved":0,"allocated":4268032,"data_stored":2353604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002162"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111987+0000","last_change":"2024-04-15T15:25:06.937502+0000","last_active":"2024-04-15T15:25:08.111987+0000","last_peered":"2024-04-15T15:25:08.111987+0000","last_clean":"2024-04-15T15:25:08.111987+0000","last_became_active":"2024-04-15T15:25:06.933540+0000","last_became_peered":"2024-04-15T15:25:06.933540+0000","last_unstale":"2024-04-15T15:25:08.111987+0000","last_undegraded":"2024-04-15T15:25:08.111987+0000","last_fullsized":"2024-04-15T15:25:08.111987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:25:18.723432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109742+0000","last_change":"2024-04-15T15:25:06.928928+0000","last_active":"2024-04-15T15:25:08.109742+0000","last_peered":"2024-04-15T15:25:08.109742+0000","last_clean":"2024-04-15T15:25:08.109742+0000","last_became_active":"2024-04-15T15:25:06.928240+0000","last_became_peered":"2024-04-15T15:25:06.928240+0000","last_unstale":"2024-04-15T15:25:08.109742+0000","last_undegraded":"2024-04-15T15:25:08.109742+0000","last_fullsized":"2024-04-15T15:25:08.109742+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:03.905713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.940242+0000","last_change":"2024-04-15T15:25:06.935862+0000","last_active":"2024-04-15T15:25:07.940242+0000","last_peered":"2024-04-15T15:25:07.940242+0000","last_clean":"2024-04-15T15:25:07.940242+0000","last_became_active":"2024-04-15T15:25:06.934512+0000","last_became_peered":"2024-04-15T15:25:06.934512+0000","last_unstale":"2024-04-15T15:25:07.940242+0000","last_undegraded":"2024-04-15T15:25:07.940242+0000","last_fullsized":"2024-04-15T15:25:07.940242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:43:24.414025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933144+0000","last_change":"2024-04-15T15:25:06.926921+0000","last_active":"2024-04-15T15:25:07.933144+0000","last_peered":"2024-04-15T15:25:07.933144+0000","last_clean":"2024-04-15T15:25:07.933144+0000","last_became_active":"2024-04-15T15:25:06.926636+0000","last_became_peered":"2024-04-15T15:25:06.926636+0000","last_unstale":"2024-04-15T15:25:07.933144+0000","last_undegraded":"2024-04-15T15:25:07.933144+0000","last_fullsized":"2024-04-15T15:25:07.933144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:31:25.313430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.935015+0000","last_change":"2024-04-15T15:25:06.929911+0000","last_active":"2024-04-15T15:25:06.935015+0000","last_peered":"2024-04-15T15:25:06.935015+0000","last_clean":"2024-04-15T15:25:06.935015+0000","last_became_active":"2024-04-15T15:25:06.926916+0000","last_became_peered":"2024-04-15T15:25:06.926916+0000","last_unstale":"2024-04-15T15:25:06.935015+0000","last_undegraded":"2024-04-15T15:25:06.935015+0000","last_fullsized":"2024-04-15T15:25:06.935015+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:48:00.129752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.108910+0000","last_change":"2024-04-15T15:25:06.928707+0000","last_active":"2024-04-15T15:25:08.108910+0000","last_peered":"2024-04-15T15:25:08.108910+0000","last_clean":"2024-04-15T15:25:08.108910+0000","last_became_active":"2024-04-15T15:25:06.928420+0000","last_became_peered":"2024-04-15T15:25:06.928420+0000","last_unstale":"2024-04-15T15:25:08.108910+0000","last_undegraded":"2024-04-15T15:25:08.108910+0000","last_fullsized":"2024-04-15T15:25:08.108910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:23:14.407218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938534+0000","last_change":"2024-04-15T15:25:06.936038+0000","last_active":"2024-04-15T15:25:07.938534+0000","last_peered":"2024-04-15T15:25:07.938534+0000","last_clean":"2024-04-15T15:25:07.938534+0000","last_became_active":"2024-04-15T15:25:06.935691+0000","last_became_peered":"2024-04-15T15:25:06.935691+0000","last_unstale":"2024-04-15T15:25:07.938534+0000","last_undegraded":"2024-04-15T15:25:07.938534+0000","last_fullsized":"2024-04-15T15:25:07.938534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:45:23.858988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937225+0000","last_change":"2024-04-15T15:25:06.933544+0000","last_active":"2024-04-15T15:25:07.937225+0000","last_peered":"2024-04-15T15:25:07.937225+0000","last_clean":"2024-04-15T15:25:07.937225+0000","last_became_active":"2024-04-15T15:25:06.933247+0000","last_became_peered":"2024-04-15T15:25:06.933247+0000","last_unstale":"2024-04-15T15:25:07.937225+0000","last_undegraded":"2024-04-15T15:25:07.937225+0000","last_fullsized":"2024-04-15T15:25:07.937225+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:12.196977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111314+0000","last_change":"2024-04-15T15:25:06.927357+0000","last_active":"2024-04-15T15:25:08.111314+0000","last_peered":"2024-04-15T15:25:08.111314+0000","last_clean":"2024-04-15T15:25:08.111314+0000","last_became_active":"2024-04-15T15:25:06.927034+0000","last_became_peered":"2024-04-15T15:25:06.927034+0000","last_unstale":"2024-04-15T15:25:08.111314+0000","last_undegraded":"2024-04-15T15:25:08.111314+0000","last_fullsized":"2024-04-15T15:25:08.111314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:42.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933865+0000","last_change":"2024-04-15T15:25:06.927238+0000","last_active":"2024-04-15T15:25:07.933865+0000","last_peered":"2024-04-15T15:25:07.933865+0000","last_clean":"2024-04-15T15:25:07.933865+0000","last_became_active":"2024-04-15T15:25:06.926927+0000","last_became_peered":"2024-04-15T15:25:06.926927+0000","last_unstale":"2024-04-15T15:25:07.933865+0000","last_undegraded":"2024-04-15T15:25:07.933865+0000","last_fullsized":"2024-04-15T15:25:07.933865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:44:23.747614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938361+0000","last_change":"2024-04-15T15:25:06.941742+0000","last_active":"2024-04-15T15:25:07.938361+0000","last_peered":"2024-04-15T15:25:07.938361+0000","last_clean":"2024-04-15T15:25:07.938361+0000","last_became_active":"2024-04-15T15:25:06.941405+0000","last_became_peered":"2024-04-15T15:25:06.941405+0000","last_unstale":"2024-04-15T15:25:07.938361+0000","last_undegraded":"2024-04-15T15:25:07.938361+0000","last_fullsized":"2024-04-15T15:25:07.938361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:43:24.414025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939355+0000","last_change":"2024-04-15T15:25:06.921853+0000","last_active":"2024-04-15T15:25:07.939355+0000","last_peered":"2024-04-15T15:25:07.939355+0000","last_clean":"2024-04-15T15:25:07.939355+0000","last_became_active":"2024-04-15T15:25:06.921491+0000","last_became_peered":"2024-04-15T15:25:06.921491+0000","last_unstale":"2024-04-15T15:25:07.939355+0000","last_undegraded":"2024-04-15T15:25:07.939355+0000","last_fullsized":"2024-04-15T15:25:07.939355+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:23:14.407218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.930089+0000","last_change":"2024-04-15T15:25:06.923195+0000","last_active":"2024-04-15T15:25:06.930089+0000","last_peered":"2024-04-15T15:25:06.930089+0000","last_clean":"2024-04-15T15:25:06.930089+0000","last_became_active":"2024-04-15T15:25:06.922872+0000","last_became_peered":"2024-04-15T15:25:06.922872+0000","last_unstale":"2024-04-15T15:25:06.930089+0000","last_undegraded":"2024-04-15T15:25:06.930089+0000","last_fullsized":"2024-04-15T15:25:06.930089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:13:33.617721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.108990+0000","last_change":"2024-04-15T15:25:06.924540+0000","last_active":"2024-04-15T15:25:08.108990+0000","last_peered":"2024-04-15T15:25:08.108990+0000","last_clean":"2024-04-15T15:25:08.108990+0000","last_became_active":"2024-04-15T15:25:06.924254+0000","last_became_peered":"2024-04-15T15:25:06.924254+0000","last_unstale":"2024-04-15T15:25:08.108990+0000","last_undegraded":"2024-04-15T15:25:08.108990+0000","last_fullsized":"2024-04-15T15:25:08.108990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:54.080186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939392+0000","last_change":"2024-04-15T15:25:06.927463+0000","last_active":"2024-04-15T15:25:07.939392+0000","last_peered":"2024-04-15T15:25:07.939392+0000","last_clean":"2024-04-15T15:25:07.939392+0000","last_became_active":"2024-04-15T15:25:06.927137+0000","last_became_peered":"2024-04-15T15:25:06.927137+0000","last_unstale":"2024-04-15T15:25:07.939392+0000","last_undegraded":"2024-04-15T15:25:07.939392+0000","last_fullsized":"2024-04-15T15:25:07.939392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:47:54.080186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938919+0000","last_change":"2024-04-15T15:25:06.940300+0000","last_active":"2024-04-15T15:25:07.938919+0000","last_peered":"2024-04-15T15:25:07.938919+0000","last_clean":"2024-04-15T15:25:07.938919+0000","last_became_active":"2024-04-15T15:25:06.940010+0000","last_became_peered":"2024-04-15T15:25:06.940010+0000","last_unstale":"2024-04-15T15:25:07.938919+0000","last_undegraded":"2024-04-15T15:25:07.938919+0000","last_fullsized":"2024-04-15T15:25:07.938919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:11:56.076782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938816+0000","last_change":"2024-04-15T15:25:06.939369+0000","last_active":"2024-04-15T15:25:07.938816+0000","last_peered":"2024-04-15T15:25:07.938816+0000","last_clean":"2024-04-15T15:25:07.938816+0000","last_became_active":"2024-04-15T15:25:06.938992+0000","last_became_peered":"2024-04-15T15:25:06.938992+0000","last_unstale":"2024-04-15T15:25:07.938816+0000","last_undegraded":"2024-04-15T15:25:07.938816+0000","last_fullsized":"2024-04-15T15:25:07.938816+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:16:12.118170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109743+0000","last_change":"2024-04-15T15:25:06.937257+0000","last_active":"2024-04-15T15:25:08.109743+0000","last_peered":"2024-04-15T15:25:08.109743+0000","last_clean":"2024-04-15T15:25:08.109743+0000","last_became_active":"2024-04-15T15:25:06.936982+0000","last_became_peered":"2024-04-15T15:25:06.936982+0000","last_unstale":"2024-04-15T15:25:08.109743+0000","last_undegraded":"2024-04-15T15:25:08.109743+0000","last_fullsized":"2024-04-15T15:25:08.109743+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:51.313422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109171+0000","last_change":"2024-04-15T15:25:06.936124+0000","last_active":"2024-04-15T15:25:08.109171+0000","last_peered":"2024-04-15T15:25:08.109171+0000","last_clean":"2024-04-15T15:25:08.109171+0000","last_became_active":"2024-04-15T15:25:06.931919+0000","last_became_peered":"2024-04-15T15:25:06.931919+0000","last_unstale":"2024-04-15T15:25:08.109171+0000","last_undegraded":"2024-04-15T15:25:08.109171+0000","last_fullsized":"2024-04-15T15:25:08.109171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:12.196977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.919464+0000","last_change":"2024-04-15T15:25:06.909188+0000","last_active":"2024-04-15T15:25:06.919464+0000","last_peered":"2024-04-15T15:25:06.919464+0000","last_clean":"2024-04-15T15:25:06.919464+0000","last_became_active":"2024-04-15T15:25:06.908430+0000","last_became_peered":"2024-04-15T15:25:06.908430+0000","last_unstale":"2024-04-15T15:25:06.919464+0000","last_undegraded":"2024-04-15T15:25:06.919464+0000","last_fullsized":"2024-04-15T15:25:06.919464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:01:01.235779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110489+0000","last_change":"2024-04-15T15:25:06.926364+0000","last_active":"2024-04-15T15:25:08.110489+0000","last_peered":"2024-04-15T15:25:08.110489+0000","last_clean":"2024-04-15T15:25:08.110489+0000","last_became_active":"2024-04-15T15:25:06.925940+0000","last_became_peered":"2024-04-15T15:25:06.925940+0000","last_unstale":"2024-04-15T15:25:08.110489+0000","last_undegraded":"2024-04-15T15:25:08.110489+0000","last_fullsized":"2024-04-15T15:25:08.110489+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:59:56.453667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110328+0000","last_change":"2024-04-15T15:25:06.929161+0000","last_active":"2024-04-15T15:25:08.110328+0000","last_peered":"2024-04-15T15:25:08.110328+0000","last_clean":"2024-04-15T15:25:08.110328+0000","last_became_active":"2024-04-15T15:25:06.927857+0000","last_became_peered":"2024-04-15T15:25:06.927857+0000","last_unstale":"2024-04-15T15:25:08.110328+0000","last_undegraded":"2024-04-15T15:25:08.110328+0000","last_fullsized":"2024-04-15T15:25:08.110328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:18:22.995620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938055+0000","last_change":"2024-04-15T15:25:06.925831+0000","last_active":"2024-04-15T15:25:07.938055+0000","last_peered":"2024-04-15T15:25:07.938055+0000","last_clean":"2024-04-15T15:25:07.938055+0000","last_became_active":"2024-04-15T15:25:06.925442+0000","last_became_peered":"2024-04-15T15:25:06.925442+0000","last_unstale":"2024-04-15T15:25:07.938055+0000","last_undegraded":"2024-04-15T15:25:07.938055+0000","last_fullsized":"2024-04-15T15:25:07.938055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:54.777764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937127+0000","last_change":"2024-04-15T15:25:06.921942+0000","last_active":"2024-04-15T15:25:07.937127+0000","last_peered":"2024-04-15T15:25:07.937127+0000","last_clean":"2024-04-15T15:25:07.937127+0000","last_became_active":"2024-04-15T15:25:06.921618+0000","last_became_peered":"2024-04-15T15:25:06.921618+0000","last_unstale":"2024-04-15T15:25:07.937127+0000","last_undegraded":"2024-04-15T15:25:07.937127+0000","last_fullsized":"2024-04-15T15:25:07.937127+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:44:23.747614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933905+0000","last_change":"2024-04-15T15:25:06.933237+0000","last_active":"2024-04-15T15:25:07.933905+0000","last_peered":"2024-04-15T15:25:07.933905+0000","last_clean":"2024-04-15T15:25:07.933905+0000","last_became_active":"2024-04-15T15:25:06.932041+0000","last_became_peered":"2024-04-15T15:25:06.932041+0000","last_unstale":"2024-04-15T15:25:07.933905+0000","last_undegraded":"2024-04-15T15:25:07.933905+0000","last_fullsized":"2024-04-15T15:25:07.933905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:23:14.407218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109849+0000","last_change":"2024-04-15T15:25:06.939659+0000","last_active":"2024-04-15T15:25:08.109849+0000","last_peered":"2024-04-15T15:25:08.109849+0000","last_clean":"2024-04-15T15:25:08.109849+0000","last_became_active":"2024-04-15T15:25:06.939087+0000","last_became_peered":"2024-04-15T15:25:06.939087+0000","last_unstale":"2024-04-15T15:25:08.109849+0000","last_undegraded":"2024-04-15T15:25:08.109849+0000","last_fullsized":"2024-04-15T15:25:08.109849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:43:24.414025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933597+0000","last_change":"2024-04-15T15:25:06.931286+0000","last_active":"2024-04-15T15:25:07.933597+0000","last_peered":"2024-04-15T15:25:07.933597+0000","last_clean":"2024-04-15T15:25:07.933597+0000","last_became_active":"2024-04-15T15:25:06.929550+0000","last_became_peered":"2024-04-15T15:25:06.929550+0000","last_unstale":"2024-04-15T15:25:07.933597+0000","last_undegraded":"2024-04-15T15:25:07.933597+0000","last_fullsized":"2024-04-15T15:25:07.933597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:46:23.614593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.940241+0000","last_change":"2024-04-15T15:25:06.913055+0000","last_active":"2024-04-15T15:25:07.940241+0000","last_peered":"2024-04-15T15:25:07.940241+0000","last_clean":"2024-04-15T15:25:07.940241+0000","last_became_active":"2024-04-15T15:25:06.912251+0000","last_became_peered":"2024-04-15T15:25:06.912251+0000","last_unstale":"2024-04-15T15:25:07.940241+0000","last_undegraded":"2024-04-15T15:25:07.940241+0000","last_fullsized":"2024-04-15T15:25:07.940241+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:51.313422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109949+0000","last_change":"2024-04-15T15:25:06.930048+0000","last_active":"2024-04-15T15:25:08.109949+0000","last_peered":"2024-04-15T15:25:08.109949+0000","last_clean":"2024-04-15T15:25:08.109949+0000","last_became_active":"2024-04-15T15:25:06.929779+0000","last_became_peered":"2024-04-15T15:25:06.929779+0000","last_unstale":"2024-04-15T15:25:08.109949+0000","last_undegraded":"2024-04-15T15:25:08.109949+0000","last_fullsized":"2024-04-15T15:25:08.109949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:52:38.724258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938063+0000","last_change":"2024-04-15T15:25:06.938089+0000","last_active":"2024-04-15T15:25:07.938063+0000","last_peered":"2024-04-15T15:25:07.938063+0000","last_clean":"2024-04-15T15:25:07.938063+0000","last_became_active":"2024-04-15T15:25:06.937742+0000","last_became_peered":"2024-04-15T15:25:06.937742+0000","last_unstale":"2024-04-15T15:25:07.938063+0000","last_undegraded":"2024-04-15T15:25:07.938063+0000","last_fullsized":"2024-04-15T15:25:07.938063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:52:48.345879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111859+0000","last_change":"2024-04-15T15:25:06.927076+0000","last_active":"2024-04-15T15:25:08.111859+0000","last_peered":"2024-04-15T15:25:08.111859+0000","last_clean":"2024-04-15T15:25:08.111859+0000","last_became_active":"2024-04-15T15:25:06.925688+0000","last_became_peered":"2024-04-15T15:25:06.925688+0000","last_unstale":"2024-04-15T15:25:08.111859+0000","last_undegraded":"2024-04-15T15:25:08.111859+0000","last_fullsized":"2024-04-15T15:25:08.111859+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:59:42.498586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937804+0000","last_change":"2024-04-15T15:25:06.937894+0000","last_active":"2024-04-15T15:25:07.937804+0000","last_peered":"2024-04-15T15:25:07.937804+0000","last_clean":"2024-04-15T15:25:07.937804+0000","last_became_active":"2024-04-15T15:25:06.937032+0000","last_became_peered":"2024-04-15T15:25:06.937032+0000","last_unstale":"2024-04-15T15:25:07.937804+0000","last_undegraded":"2024-04-15T15:25:07.937804+0000","last_fullsized":"2024-04-15T15:25:07.937804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:40.737655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938598+0000","last_change":"2024-04-15T15:25:06.930958+0000","last_active":"2024-04-15T15:25:07.938598+0000","last_peered":"2024-04-15T15:25:07.938598+0000","last_clean":"2024-04-15T15:25:07.938598+0000","last_became_active":"2024-04-15T15:25:06.930682+0000","last_became_peered":"2024-04-15T15:25:06.930682+0000","last_unstale":"2024-04-15T15:25:07.938598+0000","last_undegraded":"2024-04-15T15:25:07.938598+0000","last_fullsized":"2024-04-15T15:25:07.938598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:52:28.384976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.112309+0000","last_change":"2024-04-15T15:25:05.899821+0000","last_active":"2024-04-15T15:25:08.112309+0000","last_peered":"2024-04-15T15:25:08.112309+0000","last_clean":"2024-04-15T15:25:08.112309+0000","last_became_active":"2024-04-15T15:25:05.899522+0000","last_became_peered":"2024-04-15T15:25:05.899522+0000","last_unstale":"2024-04-15T15:25:08.112309+0000","last_undegraded":"2024-04-15T15:25:08.112309+0000","last_fullsized":"2024-04-15T15:25:08.112309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:15:40.915748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.112298+0000","last_change":"2024-04-15T15:25:06.932401+0000","last_active":"2024-04-15T15:25:08.112298+0000","last_peered":"2024-04-15T15:25:08.112298+0000","last_clean":"2024-04-15T15:25:08.112298+0000","last_became_active":"2024-04-15T15:25:06.932037+0000","last_became_peered":"2024-04-15T15:25:06.932037+0000","last_unstale":"2024-04-15T15:25:08.112298+0000","last_undegraded":"2024-04-15T15:25:08.112298+0000","last_fullsized":"2024-04-15T15:25:08.112298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:54.777764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938686+0000","last_change":"2024-04-15T15:25:05.898287+0000","last_active":"2024-04-15T15:25:07.938686+0000","last_peered":"2024-04-15T15:25:07.938686+0000","last_clean":"2024-04-15T15:25:07.938686+0000","last_became_active":"2024-04-15T15:25:05.897977+0000","last_became_peered":"2024-04-15T15:25:05.897977+0000","last_unstale":"2024-04-15T15:25:07.938686+0000","last_undegraded":"2024-04-15T15:25:07.938686+0000","last_fullsized":"2024-04-15T15:25:07.938686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:38:37.393168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110355+0000","last_change":"2024-04-15T15:25:06.931416+0000","last_active":"2024-04-15T15:25:08.110355+0000","last_peered":"2024-04-15T15:25:08.110355+0000","last_clean":"2024-04-15T15:25:08.110355+0000","last_became_active":"2024-04-15T15:25:06.930960+0000","last_became_peered":"2024-04-15T15:25:06.930960+0000","last_unstale":"2024-04-15T15:25:08.110355+0000","last_undegraded":"2024-04-15T15:25:08.110355+0000","last_fullsized":"2024-04-15T15:25:08.110355+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:52:28.384976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109294+0000","last_change":"2024-04-15T15:25:05.891999+0000","last_active":"2024-04-15T15:25:08.109294+0000","last_peered":"2024-04-15T15:25:08.109294+0000","last_clean":"2024-04-15T15:25:08.109294+0000","last_became_active":"2024-04-15T15:25:05.891707+0000","last_became_peered":"2024-04-15T15:25:05.891707+0000","last_unstale":"2024-04-15T15:25:08.109294+0000","last_undegraded":"2024-04-15T15:25:08.109294+0000","last_fullsized":"2024-04-15T15:25:08.109294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:52:37.718604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933622+0000","last_change":"2024-04-15T15:25:06.929891+0000","last_active":"2024-04-15T15:25:07.933622+0000","last_peered":"2024-04-15T15:25:07.933622+0000","last_clean":"2024-04-15T15:25:07.933622+0000","last_became_active":"2024-04-15T15:25:06.929250+0000","last_became_peered":"2024-04-15T15:25:06.929250+0000","last_unstale":"2024-04-15T15:25:07.933622+0000","last_undegraded":"2024-04-15T15:25:07.933622+0000","last_fullsized":"2024-04-15T15:25:07.933622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:24:10.537370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.945970+0000","last_change":"2024-04-15T15:25:05.898445+0000","last_active":"2024-04-15T15:25:06.945970+0000","last_peered":"2024-04-15T15:25:06.945970+0000","last_clean":"2024-04-15T15:25:06.945970+0000","last_became_active":"2024-04-15T15:25:05.898177+0000","last_became_peered":"2024-04-15T15:25:05.898177+0000","last_unstale":"2024-04-15T15:25:06.945970+0000","last_undegraded":"2024-04-15T15:25:06.945970+0000","last_fullsized":"2024-04-15T15:25:06.945970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:46:05.833643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938484+0000","last_change":"2024-04-15T15:25:06.937264+0000","last_active":"2024-04-15T15:25:07.938484+0000","last_peered":"2024-04-15T15:25:07.938484+0000","last_clean":"2024-04-15T15:25:07.938484+0000","last_became_active":"2024-04-15T15:25:06.936964+0000","last_became_peered":"2024-04-15T15:25:06.936964+0000","last_unstale":"2024-04-15T15:25:07.938484+0000","last_undegraded":"2024-04-15T15:25:07.938484+0000","last_fullsized":"2024-04-15T15:25:07.938484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T03:24:18.031910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110236+0000","last_change":"2024-04-15T15:25:05.900911+0000","last_active":"2024-04-15T15:25:08.110236+0000","last_peered":"2024-04-15T15:25:08.110236+0000","last_clean":"2024-04-15T15:25:08.110236+0000","last_became_active":"2024-04-15T15:25:05.900554+0000","last_became_peered":"2024-04-15T15:25:05.900554+0000","last_unstale":"2024-04-15T15:25:08.110236+0000","last_undegraded":"2024-04-15T15:25:08.110236+0000","last_fullsized":"2024-04-15T15:25:08.110236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:48:48.390397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939566+0000","last_change":"2024-04-15T15:25:06.915709+0000","last_active":"2024-04-15T15:25:07.939566+0000","last_peered":"2024-04-15T15:25:07.939566+0000","last_clean":"2024-04-15T15:25:07.939566+0000","last_became_active":"2024-04-15T15:25:06.914951+0000","last_became_peered":"2024-04-15T15:25:06.914951+0000","last_unstale":"2024-04-15T15:25:07.939566+0000","last_undegraded":"2024-04-15T15:25:07.939566+0000","last_fullsized":"2024-04-15T15:25:07.939566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:40.737655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110420+0000","last_change":"2024-04-15T15:25:05.901837+0000","last_active":"2024-04-15T15:25:08.110420+0000","last_peered":"2024-04-15T15:25:08.110420+0000","last_clean":"2024-04-15T15:25:08.110420+0000","last_became_active":"2024-04-15T15:25:05.901534+0000","last_became_peered":"2024-04-15T15:25:05.901534+0000","last_unstale":"2024-04-15T15:25:08.110420+0000","last_undegraded":"2024-04-15T15:25:08.110420+0000","last_fullsized":"2024-04-15T15:25:08.110420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:01:55.640226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109430+0000","last_change":"2024-04-15T15:25:06.929490+0000","last_active":"2024-04-15T15:25:08.109430+0000","last_peered":"2024-04-15T15:25:08.109430+0000","last_clean":"2024-04-15T15:25:08.109430+0000","last_became_active":"2024-04-15T15:25:06.929093+0000","last_became_peered":"2024-04-15T15:25:06.929093+0000","last_unstale":"2024-04-15T15:25:08.109430+0000","last_undegraded":"2024-04-15T15:25:08.109430+0000","last_fullsized":"2024-04-15T15:25:08.109430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:25:18.723432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.108464+0000","last_change":"2024-04-15T15:25:05.895397+0000","last_active":"2024-04-15T15:25:08.108464+0000","last_peered":"2024-04-15T15:25:08.108464+0000","last_clean":"2024-04-15T15:25:08.108464+0000","last_became_active":"2024-04-15T15:25:05.894274+0000","last_became_peered":"2024-04-15T15:25:05.894274+0000","last_unstale":"2024-04-15T15:25:08.108464+0000","last_undegraded":"2024-04-15T15:25:08.108464+0000","last_fullsized":"2024-04-15T15:25:08.108464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:04:54.969524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.932843+0000","last_change":"2024-04-15T15:25:06.921169+0000","last_active":"2024-04-15T15:25:07.932843+0000","last_peered":"2024-04-15T15:25:07.932843+0000","last_clean":"2024-04-15T15:25:07.932843+0000","last_became_active":"2024-04-15T15:25:06.920875+0000","last_became_peered":"2024-04-15T15:25:06.920875+0000","last_unstale":"2024-04-15T15:25:07.932843+0000","last_undegraded":"2024-04-15T15:25:07.932843+0000","last_fullsized":"2024-04-15T15:25:07.932843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:29:51.053513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109341+0000","last_change":"2024-04-15T15:25:05.896020+0000","last_active":"2024-04-15T15:25:08.109341+0000","last_peered":"2024-04-15T15:25:08.109341+0000","last_clean":"2024-04-15T15:25:08.109341+0000","last_became_active":"2024-04-15T15:25:05.895759+0000","last_became_peered":"2024-04-15T15:25:05.895759+0000","last_unstale":"2024-04-15T15:25:08.109341+0000","last_undegraded":"2024-04-15T15:25:08.109341+0000","last_fullsized":"2024-04-15T15:25:08.109341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:39:31.798053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939943+0000","last_change":"2024-04-15T15:25:06.928693+0000","last_active":"2024-04-15T15:25:07.939943+0000","last_peered":"2024-04-15T15:25:07.939943+0000","last_clean":"2024-04-15T15:25:07.939943+0000","last_became_active":"2024-04-15T15:25:06.928375+0000","last_became_peered":"2024-04-15T15:25:06.928375+0000","last_unstale":"2024-04-15T15:25:07.939943+0000","last_undegraded":"2024-04-15T15:25:07.939943+0000","last_fullsized":"2024-04-15T15:25:07.939943+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:52:48.345879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109811+0000","last_change":"2024-04-15T15:25:05.897944+0000","last_active":"2024-04-15T15:25:08.109811+0000","last_peered":"2024-04-15T15:25:08.109811+0000","last_clean":"2024-04-15T15:25:08.109811+0000","last_became_active":"2024-04-15T15:25:05.897634+0000","last_became_peered":"2024-04-15T15:25:05.897634+0000","last_unstale":"2024-04-15T15:25:08.109811+0000","last_undegraded":"2024-04-15T15:25:08.109811+0000","last_fullsized":"2024-04-15T15:25:08.109811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:53.772110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109669+0000","last_change":"2024-04-15T15:25:06.938843+0000","last_active":"2024-04-15T15:25:08.109669+0000","last_peered":"2024-04-15T15:25:08.109669+0000","last_clean":"2024-04-15T15:25:08.109669+0000","last_became_active":"2024-04-15T15:25:06.938560+0000","last_became_peered":"2024-04-15T15:25:06.938560+0000","last_unstale":"2024-04-15T15:25:08.109669+0000","last_undegraded":"2024-04-15T15:25:08.109669+0000","last_fullsized":"2024-04-15T15:25:08.109669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:52:48.345879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938901+0000","last_change":"2024-04-15T15:25:05.899918+0000","last_active":"2024-04-15T15:25:07.938901+0000","last_peered":"2024-04-15T15:25:07.938901+0000","last_clean":"2024-04-15T15:25:07.938901+0000","last_became_active":"2024-04-15T15:25:05.899315+0000","last_became_peered":"2024-04-15T15:25:05.899315+0000","last_unstale":"2024-04-15T15:25:07.938901+0000","last_undegraded":"2024-04-15T15:25:07.938901+0000","last_fullsized":"2024-04-15T15:25:07.938901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:57:52.551727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109250+0000","last_change":"2024-04-15T15:25:06.939452+0000","last_active":"2024-04-15T15:25:08.109250+0000","last_peered":"2024-04-15T15:25:08.109250+0000","last_clean":"2024-04-15T15:25:08.109250+0000","last_became_active":"2024-04-15T15:25:06.939106+0000","last_became_peered":"2024-04-15T15:25:06.939106+0000","last_unstale":"2024-04-15T15:25:08.109250+0000","last_undegraded":"2024-04-15T15:25:08.109250+0000","last_fullsized":"2024-04-15T15:25:08.109250+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:44:40.737655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939821+0000","last_change":"2024-04-15T15:25:05.897110+0000","last_active":"2024-04-15T15:25:07.939821+0000","last_peered":"2024-04-15T15:25:07.939821+0000","last_clean":"2024-04-15T15:25:07.939821+0000","last_became_active":"2024-04-15T15:25:05.896827+0000","last_became_peered":"2024-04-15T15:25:05.896827+0000","last_unstale":"2024-04-15T15:25:07.939821+0000","last_undegraded":"2024-04-15T15:25:07.939821+0000","last_fullsized":"2024-04-15T15:25:07.939821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:25:17.717778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939432+0000","last_change":"2024-04-15T15:25:06.935416+0000","last_active":"2024-04-15T15:25:07.939432+0000","last_peered":"2024-04-15T15:25:07.939432+0000","last_clean":"2024-04-15T15:25:07.939432+0000","last_became_active":"2024-04-15T15:25:06.934906+0000","last_became_peered":"2024-04-15T15:25:06.934906+0000","last_unstale":"2024-04-15T15:25:07.939432+0000","last_undegraded":"2024-04-15T15:25:07.939432+0000","last_fullsized":"2024-04-15T15:25:07.939432+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T15:47:12.196977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109263+0000","last_change":"2024-04-15T15:25:05.895267+0000","last_active":"2024-04-15T15:25:08.109263+0000","last_peered":"2024-04-15T15:25:08.109263+0000","last_clean":"2024-04-15T15:25:08.109263+0000","last_became_active":"2024-04-15T15:25:05.894993+0000","last_became_peered":"2024-04-15T15:25:05.894993+0000","last_unstale":"2024-04-15T15:25:08.109263+0000","last_undegraded":"2024-04-15T15:25:08.109263+0000","last_fullsized":"2024-04-15T15:25:08.109263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:25:17.717778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.932846+0000","last_change":"2024-04-15T15:25:06.922802+0000","last_active":"2024-04-15T15:25:07.932846+0000","last_peered":"2024-04-15T15:25:07.932846+0000","last_clean":"2024-04-15T15:25:07.932846+0000","last_became_active":"2024-04-15T15:25:06.922488+0000","last_became_peered":"2024-04-15T15:25:06.922488+0000","last_unstale":"2024-04-15T15:25:07.932846+0000","last_undegraded":"2024-04-15T15:25:07.932846+0000","last_fullsized":"2024-04-15T15:25:07.932846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:42.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110532+0000","last_change":"2024-04-15T15:25:05.899710+0000","last_active":"2024-04-15T15:25:08.110532+0000","last_peered":"2024-04-15T15:25:08.110532+0000","last_clean":"2024-04-15T15:25:08.110532+0000","last_became_active":"2024-04-15T15:25:05.899414+0000","last_became_peered":"2024-04-15T15:25:05.899414+0000","last_unstale":"2024-04-15T15:25:08.110532+0000","last_undegraded":"2024-04-15T15:25:08.110532+0000","last_fullsized":"2024-04-15T15:25:08.110532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:27:53.296965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109329+0000","last_change":"2024-04-15T15:25:06.939727+0000","last_active":"2024-04-15T15:25:08.109329+0000","last_peered":"2024-04-15T15:25:08.109329+0000","last_clean":"2024-04-15T15:25:08.109329+0000","last_became_active":"2024-04-15T15:25:06.936112+0000","last_became_peered":"2024-04-15T15:25:06.936112+0000","last_unstale":"2024-04-15T15:25:08.109329+0000","last_undegraded":"2024-04-15T15:25:08.109329+0000","last_fullsized":"2024-04-15T15:25:08.109329+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:52:59.487388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.932868+0000","last_change":"2024-04-15T15:25:05.892271+0000","last_active":"2024-04-15T15:25:07.932868+0000","last_peered":"2024-04-15T15:25:07.932868+0000","last_clean":"2024-04-15T15:25:07.932868+0000","last_became_active":"2024-04-15T15:25:05.890984+0000","last_became_peered":"2024-04-15T15:25:05.890984+0000","last_unstale":"2024-04-15T15:25:07.932868+0000","last_undegraded":"2024-04-15T15:25:07.932868+0000","last_fullsized":"2024-04-15T15:25:07.932868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:57:52.551727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937811+0000","last_change":"2024-04-15T15:25:06.936599+0000","last_active":"2024-04-15T15:25:07.937811+0000","last_peered":"2024-04-15T15:25:07.937811+0000","last_clean":"2024-04-15T15:25:07.937811+0000","last_became_active":"2024-04-15T15:25:06.936222+0000","last_became_peered":"2024-04-15T15:25:06.936222+0000","last_unstale":"2024-04-15T15:25:07.937811+0000","last_undegraded":"2024-04-15T15:25:07.937811+0000","last_fullsized":"2024-04-15T15:25:07.937811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:52:59.487388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.108702+0000","last_change":"2024-04-15T15:25:05.896797+0000","last_active":"2024-04-15T15:25:08.108702+0000","last_peered":"2024-04-15T15:25:08.108702+0000","last_clean":"2024-04-15T15:25:08.108702+0000","last_became_active":"2024-04-15T15:25:05.896492+0000","last_became_peered":"2024-04-15T15:25:05.896492+0000","last_unstale":"2024-04-15T15:25:08.108702+0000","last_undegraded":"2024-04-15T15:25:08.108702+0000","last_fullsized":"2024-04-15T15:25:08.108702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:20:36.568994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111378+0000","last_change":"2024-04-15T15:25:06.929759+0000","last_active":"2024-04-15T15:25:08.111378+0000","last_peered":"2024-04-15T15:25:08.111378+0000","last_clean":"2024-04-15T15:25:08.111378+0000","last_became_active":"2024-04-15T15:25:06.929466+0000","last_became_peered":"2024-04-15T15:25:06.929466+0000","last_unstale":"2024-04-15T15:25:08.111378+0000","last_undegraded":"2024-04-15T15:25:08.111378+0000","last_fullsized":"2024-04-15T15:25:08.111378+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:20:37.574648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.107768+0000","last_change":"2024-04-15T15:25:05.890106+0000","last_active":"2024-04-15T15:25:08.107768+0000","last_peered":"2024-04-15T15:25:08.107768+0000","last_clean":"2024-04-15T15:25:08.107768+0000","last_became_active":"2024-04-15T15:25:05.889812+0000","last_became_peered":"2024-04-15T15:25:05.889812+0000","last_unstale":"2024-04-15T15:25:08.107768+0000","last_undegraded":"2024-04-15T15:25:08.107768+0000","last_fullsized":"2024-04-15T15:25:08.107768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:19:07.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109869+0000","last_change":"2024-04-15T15:25:06.928928+0000","last_active":"2024-04-15T15:25:08.109869+0000","last_peered":"2024-04-15T15:25:08.109869+0000","last_clean":"2024-04-15T15:25:08.109869+0000","last_became_active":"2024-04-15T15:25:06.923616+0000","last_became_peered":"2024-04-15T15:25:06.923616+0000","last_unstale":"2024-04-15T15:25:08.109869+0000","last_undegraded":"2024-04-15T15:25:08.109869+0000","last_fullsized":"2024-04-15T15:25:08.109869+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:54.777764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.934054+0000","last_change":"2024-04-15T15:25:05.885558+0000","last_active":"2024-04-15T15:25:07.934054+0000","last_peered":"2024-04-15T15:25:07.934054+0000","last_clean":"2024-04-15T15:25:07.934054+0000","last_became_active":"2024-04-15T15:25:05.884844+0000","last_became_peered":"2024-04-15T15:25:05.884844+0000","last_unstale":"2024-04-15T15:25:07.934054+0000","last_undegraded":"2024-04-15T15:25:07.934054+0000","last_fullsized":"2024-04-15T15:25:07.934054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:20:36.568994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.939931+0000","last_change":"2024-04-15T15:25:06.924292+0000","last_active":"2024-04-15T15:25:07.939931+0000","last_peered":"2024-04-15T15:25:07.939931+0000","last_clean":"2024-04-15T15:25:07.939931+0000","last_became_active":"2024-04-15T15:25:06.923978+0000","last_became_peered":"2024-04-15T15:25:06.923978+0000","last_unstale":"2024-04-15T15:25:07.939931+0000","last_undegraded":"2024-04-15T15:25:07.939931+0000","last_fullsized":"2024-04-15T15:25:07.939931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:52:59.487388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110140+0000","last_change":"2024-04-15T15:25:05.902120+0000","last_active":"2024-04-15T15:25:08.110140+0000","last_peered":"2024-04-15T15:25:08.110140+0000","last_clean":"2024-04-15T15:25:08.110140+0000","last_became_active":"2024-04-15T15:25:05.901793+0000","last_became_peered":"2024-04-15T15:25:05.901793+0000","last_unstale":"2024-04-15T15:25:08.110140+0000","last_undegraded":"2024-04-15T15:25:08.110140+0000","last_fullsized":"2024-04-15T15:25:08.110140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:46:31.178019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933408+0000","last_change":"2024-04-15T15:25:06.921502+0000","last_active":"2024-04-15T15:25:07.933408+0000","last_peered":"2024-04-15T15:25:07.933408+0000","last_clean":"2024-04-15T15:25:07.933408+0000","last_became_active":"2024-04-15T15:25:06.921155+0000","last_became_peered":"2024-04-15T15:25:06.921155+0000","last_unstale":"2024-04-15T15:25:07.933408+0000","last_undegraded":"2024-04-15T15:25:07.933408+0000","last_fullsized":"2024-04-15T15:25:07.933408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:29:56.797123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110483+0000","last_change":"2024-04-15T15:25:05.903279+0000","last_active":"2024-04-15T15:25:08.110483+0000","last_peered":"2024-04-15T15:25:08.110483+0000","last_clean":"2024-04-15T15:25:08.110483+0000","last_became_active":"2024-04-15T15:25:05.902981+0000","last_became_peered":"2024-04-15T15:25:05.902981+0000","last_unstale":"2024-04-15T15:25:08.110483+0000","last_undegraded":"2024-04-15T15:25:08.110483+0000","last_fullsized":"2024-04-15T15:25:08.110483+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:28.015619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.934487+0000","last_change":"2024-04-15T15:25:06.926613+0000","last_active":"2024-04-15T15:25:06.934487+0000","last_peered":"2024-04-15T15:25:06.934487+0000","last_clean":"2024-04-15T15:25:06.934487+0000","last_became_active":"2024-04-15T15:25:06.926347+0000","last_became_peered":"2024-04-15T15:25:06.926347+0000","last_unstale":"2024-04-15T15:25:06.934487+0000","last_undegraded":"2024-04-15T15:25:06.934487+0000","last_fullsized":"2024-04-15T15:25:06.934487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:42:22.989173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110635+0000","last_change":"2024-04-15T15:25:05.900982+0000","last_active":"2024-04-15T15:25:08.110635+0000","last_peered":"2024-04-15T15:25:08.110635+0000","last_clean":"2024-04-15T15:25:08.110635+0000","last_became_active":"2024-04-15T15:25:05.900568+0000","last_became_peered":"2024-04-15T15:25:05.900568+0000","last_unstale":"2024-04-15T15:25:08.110635+0000","last_undegraded":"2024-04-15T15:25:08.110635+0000","last_fullsized":"2024-04-15T15:25:08.110635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:03:56.673616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.931264+0000","last_change":"2024-04-15T15:25:06.925851+0000","last_active":"2024-04-15T15:25:06.931264+0000","last_peered":"2024-04-15T15:25:06.931264+0000","last_clean":"2024-04-15T15:25:06.931264+0000","last_became_active":"2024-04-15T15:25:06.923472+0000","last_became_peered":"2024-04-15T15:25:06.923472+0000","last_unstale":"2024-04-15T15:25:06.931264+0000","last_undegraded":"2024-04-15T15:25:06.931264+0000","last_fullsized":"2024-04-15T15:25:06.931264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:51:37.853716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938257+0000","last_change":"2024-04-15T15:25:05.891567+0000","last_active":"2024-04-15T15:25:07.938257+0000","last_peered":"2024-04-15T15:25:07.938257+0000","last_clean":"2024-04-15T15:25:07.938257+0000","last_became_active":"2024-04-15T15:25:05.890204+0000","last_became_peered":"2024-04-15T15:25:05.890204+0000","last_unstale":"2024-04-15T15:25:07.938257+0000","last_undegraded":"2024-04-15T15:25:07.938257+0000","last_fullsized":"2024-04-15T15:25:07.938257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:15:40.915748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.921767+0000","last_change":"2024-04-15T15:25:06.909790+0000","last_active":"2024-04-15T15:25:06.921767+0000","last_peered":"2024-04-15T15:25:06.921767+0000","last_clean":"2024-04-15T15:25:06.921767+0000","last_became_active":"2024-04-15T15:25:06.908954+0000","last_became_peered":"2024-04-15T15:25:06.908954+0000","last_unstale":"2024-04-15T15:25:06.921767+0000","last_undegraded":"2024-04-15T15:25:06.921767+0000","last_fullsized":"2024-04-15T15:25:06.921767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:32:20.401368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111469+0000","last_change":"2024-04-15T15:25:05.896334+0000","last_active":"2024-04-15T15:25:08.111469+0000","last_peered":"2024-04-15T15:25:08.111469+0000","last_clean":"2024-04-15T15:25:08.111469+0000","last_became_active":"2024-04-15T15:25:05.896051+0000","last_became_peered":"2024-04-15T15:25:05.896051+0000","last_unstale":"2024-04-15T15:25:08.111469+0000","last_undegraded":"2024-04-15T15:25:08.111469+0000","last_fullsized":"2024-04-15T15:25:08.111469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T22:11:28.015619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.921964+0000","last_change":"2024-04-15T15:25:06.911563+0000","last_active":"2024-04-15T15:25:06.921964+0000","last_peered":"2024-04-15T15:25:06.921964+0000","last_clean":"2024-04-15T15:25:06.921964+0000","last_became_active":"2024-04-15T15:25:06.910792+0000","last_became_peered":"2024-04-15T15:25:06.910792+0000","last_unstale":"2024-04-15T15:25:06.921964+0000","last_undegraded":"2024-04-15T15:25:06.921964+0000","last_fullsized":"2024-04-15T15:25:06.921964+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T19:04:51.287497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109440+0000","last_change":"2024-04-15T15:25:05.903187+0000","last_active":"2024-04-15T15:25:08.109440+0000","last_peered":"2024-04-15T15:25:08.109440+0000","last_clean":"2024-04-15T15:25:08.109440+0000","last_became_active":"2024-04-15T15:25:05.900973+0000","last_became_peered":"2024-04-15T15:25:05.900973+0000","last_unstale":"2024-04-15T15:25:08.109440+0000","last_undegraded":"2024-04-15T15:25:08.109440+0000","last_fullsized":"2024-04-15T15:25:08.109440+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T21:30:40.996483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938083+0000","last_change":"2024-04-15T15:25:06.931216+0000","last_active":"2024-04-15T15:25:07.938083+0000","last_peered":"2024-04-15T15:25:07.938083+0000","last_clean":"2024-04-15T15:25:07.938083+0000","last_became_active":"2024-04-15T15:25:06.930712+0000","last_became_peered":"2024-04-15T15:25:06.930712+0000","last_unstale":"2024-04-15T15:25:07.938083+0000","last_undegraded":"2024-04-15T15:25:07.938083+0000","last_fullsized":"2024-04-15T15:25:07.938083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:03.905713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.937375+0000","last_change":"2024-04-15T15:25:05.900424+0000","last_active":"2024-04-15T15:25:07.937375+0000","last_peered":"2024-04-15T15:25:07.937375+0000","last_clean":"2024-04-15T15:25:07.937375+0000","last_became_active":"2024-04-15T15:25:05.900139+0000","last_became_peered":"2024-04-15T15:25:05.900139+0000","last_unstale":"2024-04-15T15:25:07.937375+0000","last_undegraded":"2024-04-15T15:25:07.937375+0000","last_fullsized":"2024-04-15T15:25:07.937375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T20:34:53.772110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-15T15:25:06.918132+0000","last_change":"2024-04-15T15:25:06.909587+0000","last_active":"2024-04-15T15:25:06.918132+0000","last_peered":"2024-04-15T15:25:06.918132+0000","last_clean":"2024-04-15T15:25:06.918132+0000","last_became_active":"2024-04-15T15:25:06.908974+0000","last_became_peered":"2024-04-15T15:25:06.908974+0000","last_unstale":"2024-04-15T15:25:06.918132+0000","last_undegraded":"2024-04-15T15:25:06.918132+0000","last_fullsized":"2024-04-15T15:25:06.918132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:53:31.878708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.934063+0000","last_change":"2024-04-15T15:25:05.880651+0000","last_active":"2024-04-15T15:25:07.934063+0000","last_peered":"2024-04-15T15:25:07.934063+0000","last_clean":"2024-04-15T15:25:07.934063+0000","last_became_active":"2024-04-15T15:25:05.880298+0000","last_became_peered":"2024-04-15T15:25:05.880298+0000","last_unstale":"2024-04-15T15:25:07.934063+0000","last_undegraded":"2024-04-15T15:25:07.934063+0000","last_fullsized":"2024-04-15T15:25:07.934063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:19:07.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.940005+0000","last_change":"2024-04-15T15:25:06.922016+0000","last_active":"2024-04-15T15:25:07.940005+0000","last_peered":"2024-04-15T15:25:07.940005+0000","last_clean":"2024-04-15T15:25:07.940005+0000","last_became_active":"2024-04-15T15:25:06.921260+0000","last_became_peered":"2024-04-15T15:25:06.921260+0000","last_unstale":"2024-04-15T15:25:07.940005+0000","last_undegraded":"2024-04-15T15:25:07.940005+0000","last_fullsized":"2024-04-15T15:25:07.940005+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:03.905713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.942056+0000","last_change":"2024-04-15T15:25:05.890841+0000","last_active":"2024-04-15T15:25:07.942056+0000","last_peered":"2024-04-15T15:25:07.942056+0000","last_clean":"2024-04-15T15:25:07.942056+0000","last_became_active":"2024-04-15T15:25:05.889042+0000","last_became_peered":"2024-04-15T15:25:05.889042+0000","last_unstale":"2024-04-15T15:25:07.942056+0000","last_undegraded":"2024-04-15T15:25:07.942056+0000","last_fullsized":"2024-04-15T15:25:07.942056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:42.892474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938369+0000","last_change":"2024-04-15T15:25:06.932945+0000","last_active":"2024-04-15T15:25:07.938369+0000","last_peered":"2024-04-15T15:25:07.938369+0000","last_clean":"2024-04-15T15:25:07.938369+0000","last_became_active":"2024-04-15T15:25:06.931880+0000","last_became_peered":"2024-04-15T15:25:06.931880+0000","last_unstale":"2024-04-15T15:25:07.938369+0000","last_undegraded":"2024-04-15T15:25:07.938369+0000","last_fullsized":"2024-04-15T15:25:07.938369+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:18:22.995620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.933407+0000","last_change":"2024-04-15T15:25:05.894631+0000","last_active":"2024-04-15T15:25:07.933407+0000","last_peered":"2024-04-15T15:25:07.933407+0000","last_clean":"2024-04-15T15:25:07.933407+0000","last_became_active":"2024-04-15T15:25:05.894303+0000","last_became_peered":"2024-04-15T15:25:05.894303+0000","last_unstale":"2024-04-15T15:25:07.933407+0000","last_undegraded":"2024-04-15T15:25:07.933407+0000","last_fullsized":"2024-04-15T15:25:07.933407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:04:54.969524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111501+0000","last_change":"2024-04-15T15:25:06.933286+0000","last_active":"2024-04-15T15:25:08.111501+0000","last_peered":"2024-04-15T15:25:08.111501+0000","last_clean":"2024-04-15T15:25:08.111501+0000","last_became_active":"2024-04-15T15:25:06.932559+0000","last_became_peered":"2024-04-15T15:25:06.932559+0000","last_unstale":"2024-04-15T15:25:08.111501+0000","last_undegraded":"2024-04-15T15:25:08.111501+0000","last_fullsized":"2024-04-15T15:25:08.111501+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:19:08.636594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938445+0000","last_change":"2024-04-15T15:25:05.892283+0000","last_active":"2024-04-15T15:25:07.938445+0000","last_peered":"2024-04-15T15:25:07.938445+0000","last_clean":"2024-04-15T15:25:07.938445+0000","last_became_active":"2024-04-15T15:25:05.891991+0000","last_became_peered":"2024-04-15T15:25:05.891991+0000","last_unstale":"2024-04-15T15:25:07.938445+0000","last_undegraded":"2024-04-15T15:25:07.938445+0000","last_fullsized":"2024-04-15T15:25:07.938445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:18:18.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938360+0000","last_change":"2024-04-15T15:25:06.932407+0000","last_active":"2024-04-15T15:25:07.938360+0000","last_peered":"2024-04-15T15:25:07.938360+0000","last_clean":"2024-04-15T15:25:07.938360+0000","last_became_active":"2024-04-15T15:25:06.931130+0000","last_became_peered":"2024-04-15T15:25:06.931130+0000","last_unstale":"2024-04-15T15:25:07.938360+0000","last_undegraded":"2024-04-15T15:25:07.938360+0000","last_fullsized":"2024-04-15T15:25:07.938360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:52:38.724258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.932984+0000","last_change":"2024-04-15T15:25:05.880000+0000","last_active":"2024-04-15T15:25:07.932984+0000","last_peered":"2024-04-15T15:25:07.932984+0000","last_clean":"2024-04-15T15:25:07.932984+0000","last_became_active":"2024-04-15T15:25:05.879548+0000","last_became_peered":"2024-04-15T15:25:05.879548+0000","last_unstale":"2024-04-15T15:25:07.932984+0000","last_undegraded":"2024-04-15T15:25:07.932984+0000","last_fullsized":"2024-04-15T15:25:07.932984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T02:12:42.892474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.107687+0000","last_change":"2024-04-15T15:25:06.927133+0000","last_active":"2024-04-15T15:25:08.107687+0000","last_peered":"2024-04-15T15:25:08.107687+0000","last_clean":"2024-04-15T15:25:08.107687+0000","last_became_active":"2024-04-15T15:25:06.926840+0000","last_became_peered":"2024-04-15T15:25:06.926840+0000","last_unstale":"2024-04-15T15:25:08.107687+0000","last_undegraded":"2024-04-15T15:25:08.107687+0000","last_fullsized":"2024-04-15T15:25:08.107687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T00:44:23.747614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.109706+0000","last_change":"2024-04-15T15:25:05.902440+0000","last_active":"2024-04-15T15:25:08.109706+0000","last_peered":"2024-04-15T15:25:08.109706+0000","last_clean":"2024-04-15T15:25:08.109706+0000","last_became_active":"2024-04-15T15:25:05.901490+0000","last_became_peered":"2024-04-15T15:25:05.901490+0000","last_unstale":"2024-04-15T15:25:08.109706+0000","last_undegraded":"2024-04-15T15:25:08.109706+0000","last_fullsized":"2024-04-15T15:25:08.109706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T23:30:10.254803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.110632+0000","last_change":"2024-04-15T15:25:06.929366+0000","last_active":"2024-04-15T15:25:08.110632+0000","last_peered":"2024-04-15T15:25:08.110632+0000","last_clean":"2024-04-15T15:25:08.110632+0000","last_became_active":"2024-04-15T15:25:06.928041+0000","last_became_peered":"2024-04-15T15:25:06.928041+0000","last_unstale":"2024-04-15T15:25:08.110632+0000","last_undegraded":"2024-04-15T15:25:08.110632+0000","last_fullsized":"2024-04-15T15:25:08.110632+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:36:38.283098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.107835+0000","last_change":"2024-04-15T15:25:05.900976+0000","last_active":"2024-04-15T15:25:08.107835+0000","last_peered":"2024-04-15T15:25:08.107835+0000","last_clean":"2024-04-15T15:25:08.107835+0000","last_became_active":"2024-04-15T15:25:05.900780+0000","last_became_peered":"2024-04-15T15:25:05.900780+0000","last_unstale":"2024-04-15T15:25:08.107835+0000","last_undegraded":"2024-04-15T15:25:08.107835+0000","last_fullsized":"2024-04-15T15:25:08.107835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:59:41.492932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938315+0000","last_change":"2024-04-15T15:25:06.934205+0000","last_active":"2024-04-15T15:25:07.938315+0000","last_peered":"2024-04-15T15:25:07.938315+0000","last_clean":"2024-04-15T15:25:07.938315+0000","last_became_active":"2024-04-15T15:25:06.933881+0000","last_became_peered":"2024-04-15T15:25:06.933881+0000","last_unstale":"2024-04-15T15:25:07.938315+0000","last_undegraded":"2024-04-15T15:25:07.938315+0000","last_fullsized":"2024-04-15T15:25:07.938315+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:05.853236+0000","last_clean_scrub_stamp":"2024-04-15T15:25:05.853236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T01:31:51.313422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:08.111424+0000","last_change":"2024-04-15T15:25:05.905016+0000","last_active":"2024-04-15T15:25:08.111424+0000","last_peered":"2024-04-15T15:25:08.111424+0000","last_clean":"2024-04-15T15:25:08.111424+0000","last_became_active":"2024-04-15T15:25:05.904813+0000","last_became_peered":"2024-04-15T15:25:05.904813+0000","last_unstale":"2024-04-15T15:25:08.111424+0000","last_undegraded":"2024-04-15T15:25:08.111424+0000","last_fullsized":"2024-04-15T15:25:08.111424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:25:04.847582+0000","last_clean_scrub_stamp":"2024-04-15T15:25:04.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T18:03:56.673616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-15T15:25:07.938252+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:25:07.938252+0000","last_peered":"2024-04-15T15:25:07.938252+0000","last_clean":"2024-04-15T15:25:07.938252+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:25:07.938252+0000","last_undegraded":"2024-04-15T15:25:07.938252+0000","last_fullsized":"2024-04-15T15:25:07.938252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":736464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":146622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":146110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":147640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":147640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":736374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:25:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-15T15:25:11.897 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-15T15:25:12.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.157+0000 7f2afa400640 1 Processor -- start 2024-04-15T15:25:12.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.158+0000 7f2afa400640 1 -- start start 2024-04-15T15:25:12.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.159+0000 7f2afa400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2af406fc80 0x7f2af415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.159+0000 7f2afa400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2af4072150 0x7f2af4072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.159+0000 7f2afa400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af415d520 con 0x7f2af415a440 2024-04-15T15:25:12.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.159+0000 7f2afa400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f2af406e280 con 0x7f2af4072150 2024-04-15T15:25:12.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.159+0000 7f2afa400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2af4072ad0 con 0x7f2af406fc80 2024-04-15T15:25:12.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.160+0000 7f2af3e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2af4072150 0x7f2af4072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.160+0000 7f2af3e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2af4072150 0x7f2af4072590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55252/0 (socket says 172.21.15.150:55252) 2024-04-15T15:25:12.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.160+0000 7f2af3e00640 1 -- 172.21.15.150:0/4052232108 learned_addr learned my addr 172.21.15.150:0/4052232108 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:12.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.160+0000 7f2af2a00640 1 --2- 172.21.15.150:0/4052232108 >> v2:172.21.15.187:3300/0 conn(0x7f2af406fc80 0x7f2af415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.160+0000 7f2af3e00640 1 -- 172.21.15.150:0/4052232108 >> v2:172.21.15.187:3300/0 conn(0x7f2af406fc80 msgr2=0x7f2af415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:12.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.160+0000 7f2af3e00640 1 --2- 172.21.15.150:0/4052232108 >> v2:172.21.15.187:3300/0 conn(0x7f2af406fc80 0x7f2af415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.160+0000 7f2af3e00640 1 -- 172.21.15.150:0/4052232108 >> v1:172.21.15.150:6789/0 conn(0x7f2af415a440 legacy=0x7f2af40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.160+0000 7f2af3e00640 1 -- 172.21.15.150:0/4052232108 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af415d800 con 0x7f2af4072150 2024-04-15T15:25:12.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.161+0000 7f2af3e00640 1 --2- 172.21.15.150:0/4052232108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4072150 0x7f2af4072590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2af40ccb70 tx=0x7f2aec02b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ca8fa7021154be server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2af2000640 1 -- 172.21.15.150:0/4052232108 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec034070 con 0x7f2af4072150 2024-04-15T15:25:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2af2000640 1 -- 172.21.15.150:0/4052232108 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec030880 con 0x7f2af4072150 2024-04-15T15:25:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2af2000640 1 -- 172.21.15.150:0/4052232108 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec030c50 con 0x7f2af4072150 2024-04-15T15:25:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2afa400640 1 -- 172.21.15.150:0/4052232108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4072150 msgr2=0x7f2af4072590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2afa400640 1 --2- 172.21.15.150:0/4052232108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4072150 0x7f2af4072590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2af40ccb70 tx=0x7f2aec02b9c0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2afa400640 1 -- 172.21.15.150:0/4052232108 shutdown_connections 2024-04-15T15:25:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2afa400640 1 --2- 172.21.15.150:0/4052232108 >> v2:172.21.15.187:3300/0 conn(0x7f2af406fc80 0x7f2af415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2afa400640 1 --2- 172.21.15.150:0/4052232108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4072150 0x7f2af4072590 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2afa400640 1 -- 172.21.15.150:0/4052232108 >> 172.21.15.150:0/4052232108 conn(0x7f2af40b2460 msgr2=0x7f2af40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:12.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2afa400640 1 -- 172.21.15.150:0/4052232108 shutdown_connections 2024-04-15T15:25:12.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.162+0000 7f2afa400640 1 -- 172.21.15.150:0/4052232108 wait complete. 2024-04-15T15:25:12.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.164+0000 7f2afa400640 1 Processor -- start 2024-04-15T15:25:12.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.164+0000 7f2afa400640 1 -- start start 2024-04-15T15:25:12.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.165+0000 7f2afa400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af406fc80 0x7f2af413ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.165+0000 7f2afa400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2af4072150 0x7f2af40ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.165+0000 7f2afa400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af415a440 0x7f2af40abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.165+0000 7f2afa400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2af415a820 con 0x7f2af415a440 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.165+0000 7f2afa400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2af40cd7f0 con 0x7f2af406fc80 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.165+0000 7f2afa400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2af415ddc0 con 0x7f2af4072150 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af415a440 0x7f2af40abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af415a440 0x7f2af40abee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55264/0 (socket says 172.21.15.150:55264) 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 -- 172.21.15.150:0/2235029578 learned_addr learned my addr 172.21.15.150:0/2235029578 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af406fc80 0x7f2af413ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af2a00640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2af4072150 0x7f2af40ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 -- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2af4072150 msgr2=0x7f2af40ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2af4072150 0x7f2af40ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 -- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af406fc80 msgr2=0x7f2af413ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af406fc80 0x7f2af413ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 -- 172.21.15.150:0/2235029578 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af40adbe0 con 0x7f2af415a440 2024-04-15T15:25:12.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.166+0000 7f2af3e00640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af415a440 0x7f2af40abee0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2aec02ba00 tx=0x7f2aec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:12.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.167+0000 7f2aeb400640 1 -- 172.21.15.150:0/2235029578 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec048070 con 0x7f2af415a440 2024-04-15T15:25:12.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.167+0000 7f2aeb400640 1 -- 172.21.15.150:0/2235029578 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec039d10 con 0x7f2af415a440 2024-04-15T15:25:12.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.167+0000 7f2aeb400640 1 -- 172.21.15.150:0/2235029578 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec034040 con 0x7f2af415a440 2024-04-15T15:25:12.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.167+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af40adec0 con 0x7f2af415a440 2024-04-15T15:25:12.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.167+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af40ae480 con 0x7f2af415a440 2024-04-15T15:25:12.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.168+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af4068c80 con 0x7f2af415a440 2024-04-15T15:25:12.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.169+0000 7f2aeb400640 1 -- 172.21.15.150:0/2235029578 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2aec04b070 con 0x7f2af415a440 2024-04-15T15:25:12.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.170+0000 7f2aeb400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ad006c8f0 0x7f2ad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.170+0000 7f2aeb400640 1 -- 172.21.15.150:0/2235029578 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2aec0ade10 con 0x7f2af415a440 2024-04-15T15:25:12.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.171+0000 7f2af3400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ad006c8f0 0x7f2ad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.172+0000 7f2af3400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ad006c8f0 0x7f2ad006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4007fd0 tx=0x7f2ae4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:12.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.174+0000 7f2aeb400640 1 -- 172.21.15.150:0/2235029578 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2aec07c2d0 con 0x7f2af415a440 2024-04-15T15:25:12.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.447+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2af40732a0 con 0x7f2af415a440 2024-04-15T15:25:12.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.453+0000 7f2aeb400640 1 -- 172.21.15.150:0/2235029578 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f2aec07bbd0 con 0x7f2af415a440 2024-04-15T15:25:12.455 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-04-15T15:25:12.455 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBLRh1mMhWcIhAALcn3OkkYUWxiZFNe4T1a0A== 2024-04-15T15:25:12.455 INFO:teuthology.orchestra.run.smithi150.stdout: caps mds = "allow" 2024-04-15T15:25:12.455 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "allow r" 2024-04-15T15:25:12.455 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-15T15:25:12.455 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.0 2024-04-15T15:25:12.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.455+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ad006c8f0 msgr2=0x7f2ad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.455+0000 7f2afa400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ad006c8f0 0x7f2ad006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4007fd0 tx=0x7f2ae4009040 comp rx=0 tx=0).stop 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.455+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af415a440 msgr2=0x7f2af40abee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.455+0000 7f2afa400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af415a440 0x7f2af40abee0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2aec02ba00 tx=0x7f2aec004730 comp rx=0 tx=0).stop 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.455+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 shutdown_connections 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.456+0000 7f2afa400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af415a440 0x7f2af40abee0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.456+0000 7f2afa400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2ad006c8f0 0x7f2ad006edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.456+0000 7f2afa400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2af4072150 0x7f2af40ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.456+0000 7f2afa400640 1 --2- 172.21.15.150:0/2235029578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af406fc80 0x7f2af413ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.456+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 >> 172.21.15.150:0/2235029578 conn(0x7f2af40b2460 msgr2=0x7f2af415f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.456+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 shutdown_connections 2024-04-15T15:25:12.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.456+0000 7f2afa400640 1 -- 172.21.15.150:0/2235029578 wait complete. 2024-04-15T15:25:12.476 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:12.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.682+0000 7fa33f200640 1 Processor -- start 2024-04-15T15:25:12.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.682+0000 7fa33f200640 1 -- start start 2024-04-15T15:25:12.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa33f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 0x7fa3380ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa33f200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa3380b3fe0 0x7fa33815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa33f200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3380b4be0 con 0x7fa3380cedf0 2024-04-15T15:25:12.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa33f200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa3380a9640 con 0x7fa3380b3fe0 2024-04-15T15:25:12.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa33f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa3380cf250 con 0x7fa3381461f0 2024-04-15T15:25:12.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa337400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa3380cedf0 0x7fa3380b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:55202/0 (socket says 172.21.15.150:55202) 2024-04-15T15:25:12.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa337400640 1 -- 172.21.15.150:0/1136512131 learned_addr learned my addr 172.21.15.150:0/1136512131 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa337e00640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 0x7fa3380ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.684+0000 7fa33d200640 1 --2- 172.21.15.150:0/1136512131 >> v2:172.21.15.187:3300/0 conn(0x7fa3380b3fe0 0x7fa33815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2207471693 0 0) 0x7fa3380b4be0 con 0x7fa3380cedf0 2024-04-15T15:25:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa31c003520 con 0x7fa3380cedf0 2024-04-15T15:25:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3117693254 0 0) 0x7fa31c003520 con 0x7fa3380cedf0 2024-04-15T15:25:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 msgr2=0x7fa3380ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 0x7fa3380ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 >> v2:172.21.15.187:3300/0 conn(0x7fa3380b3fe0 msgr2=0x7fa33815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 --2- 172.21.15.150:0/1136512131 >> v2:172.21.15.187:3300/0 conn(0x7fa3380b3fe0 0x7fa33815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3380b4620 con 0x7fa3380cedf0 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fa324001670 con 0x7fa3380cedf0 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 >> v1:172.21.15.150:6789/0 conn(0x7fa3380cedf0 legacy=0x7fa3380b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c0039b0 0x7fa31c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa31c004800 0x7fa31c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 0x7fa31c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c00b9c0 con 0x7fa31c004800 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c00bb40 con 0x7fa31c0039b0 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c00bcc0 con 0x7fa3381461f0 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.685+0000 7fa337400640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa31c004800 0x7fa31c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.686+0000 7fa337e00640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c0039b0 0x7fa31c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.686+0000 7fa33d200640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 0x7fa31c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.686+0000 7fa337400640 1 -- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 msgr2=0x7fa31c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:12.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.686+0000 7fa337400640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 0x7fa31c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.686+0000 7fa337400640 1 -- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c0039b0 msgr2=0x7fa31c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:12.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.686+0000 7fa337400640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c0039b0 0x7fa31c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.686+0000 7fa337400640 1 -- 172.21.15.150:0/1136512131 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c003520 con 0x7fa31c004800 2024-04-15T15:25:12.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.686+0000 7fa33d200640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3381461f0 0x7fa31c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:12.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.687+0000 7fa337400640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa31c004800 0x7fa31c004c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c110 tx=0x7fa31c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4c819e60724201 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:12.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.687+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324007c40 con 0x7fa31c004800 2024-04-15T15:25:12.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.687+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa324030070 con 0x7fa31c004800 2024-04-15T15:25:12.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.687+0000 7fa336a00640 1 -- 172.21.15.150:0/1136512131 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324005d30 con 0x7fa31c004800 2024-04-15T15:25:12.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.688+0000 7fa33f200640 1 -- 172.21.15.150:0/1136512131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa31c004800 msgr2=0x7fa31c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.688+0000 7fa33f200640 1 --2- 172.21.15.150:0/1136512131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa31c004800 0x7fa31c004c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c110 tx=0x7fa31c005830 comp rx=0 tx=0).stop 2024-04-15T15:25:12.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.688+0000 7fa33d200640 1 -- 172.21.15.150:0/1136512131 reap_dead start 2024-04-15T15:25:12.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.688+0000 7fa33f200640 1 -- 172.21.15.150:0/1136512131 shutdown_connections 2024-04-15T15:25:12.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.688+0000 7fa33f200640 1 -- 172.21.15.150:0/1136512131 >> 172.21.15.150:0/1136512131 conn(0x7fa3380b27b0 msgr2=0x7fa3380b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:12.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.688+0000 7fa33f200640 1 -- 172.21.15.150:0/1136512131 shutdown_connections 2024-04-15T15:25:12.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.688+0000 7fa33f200640 1 -- 172.21.15.150:0/1136512131 wait complete. 2024-04-15T15:25:12.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.690+0000 7fa33f200640 1 Processor -- start 2024-04-15T15:25:12.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.690+0000 7fa33f200640 1 -- start start 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa31c004800 0x7fa33813b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa33813e080 0x7fa33813e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa338141500 0x7fa338141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa33815d290 con 0x7fa338141500 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa33815cfb0 con 0x7fa33813e080 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa3381465d0 con 0x7fa31c004800 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa338141500 0x7fa338141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa338141500 0x7fa338141980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55282/0 (socket says 172.21.15.150:55282) 2024-04-15T15:25:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.691+0000 7fa33d200640 1 -- 172.21.15.150:0/359088238 learned_addr learned my addr 172.21.15.150:0/359088238 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa337e00640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa31c004800 0x7fa33813b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa337400640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa33813e080 0x7fa33813e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa33d200640 1 -- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa31c004800 msgr2=0x7fa33813b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa33d200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa31c004800 0x7fa33813b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa33d200640 1 -- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa33813e080 msgr2=0x7fa33813e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa33d200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa33813e080 0x7fa33813e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa33d200640 1 -- 172.21.15.150:0/359088238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa338142760 con 0x7fa338141500 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa337400640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa33813e080 0x7fa33813e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa33d200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa338141500 0x7fa338141980 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3380cf9d0 tx=0x7fa33000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa32fe00640 1 -- 172.21.15.150:0/359088238 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33001c070 con 0x7fa338141500 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa32fe00640 1 -- 172.21.15.150:0/359088238 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa330017400 con 0x7fa338141500 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3381421a0 con 0x7fa338141500 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa32fe00640 1 -- 172.21.15.150:0/359088238 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300177d0 con 0x7fa338141500 2024-04-15T15:25:12.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.692+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3380adc50 con 0x7fa338141500 2024-04-15T15:25:12.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.694+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3380d0000 con 0x7fa338141500 2024-04-15T15:25:12.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.695+0000 7fa32fe00640 1 -- 172.21.15.150:0/359088238 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa330005d10 con 0x7fa338141500 2024-04-15T15:25:12.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.695+0000 7fa32fe00640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa31806c920 0x7fa31806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:12.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.696+0000 7fa32fe00640 1 -- 172.21.15.150:0/359088238 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa33008be20 con 0x7fa338141500 2024-04-15T15:25:12.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.696+0000 7fa337e00640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa31806c920 0x7fa31806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:12.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.697+0000 7fa337e00640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa31806c920 0x7fa31806ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3380703f0 tx=0x7fa328008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:12.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.701+0000 7fa32fe00640 1 -- 172.21.15.150:0/359088238 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3380d0000 con 0x7fa338141500 2024-04-15T15:25:12.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.944+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3380cfa40 con 0x7fa338141500 2024-04-15T15:25:12.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.945+0000 7fa32fe00640 1 -- 172.21.15.150:0/359088238 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fa3380cfa40 con 0x7fa338141500 2024-04-15T15:25:12.947 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:12.947 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:09.435623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6835/3327940880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3327940880},{"type":"v1","addr":"172.21.15.150:6835","nonce":3327940880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-15T15:25:12.947 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 17 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa31806c920 msgr2=0x7fa31806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa31806c920 0x7fa31806ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3380703f0 tx=0x7fa328008040 comp rx=0 tx=0).stop 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa338141500 msgr2=0x7fa338141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa338141500 0x7fa338141980 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3380cf9d0 tx=0x7fa33000f4a0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 shutdown_connections 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa338141500 0x7fa338141980 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa33813e080 0x7fa33813e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa31806c920 0x7fa31806ede0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 --2- 172.21.15.150:0/359088238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa31c004800 0x7fa33813b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:12.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 >> 172.21.15.150:0/359088238 conn(0x7fa3380b27b0 msgr2=0x7fa3380b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:12.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 shutdown_connections 2024-04-15T15:25:12.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:12.948+0000 7fa33f200640 1 -- 172.21.15.150:0/359088238 wait complete. 2024-04-15T15:25:13.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:25:06.343634+0000', 'modified': '2024-04-15T15:25:09.435623+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5112}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5112': {'gid': 5112, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.150:6835/3327940880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 3327940880}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 3327940880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5112, 'qdb_cluster': [5112]} 2024-04-15T15:25:13.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:25:13.373 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-15T15:25:13.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.587+0000 7f8043200640 1 Processor -- start 2024-04-15T15:25:13.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.587+0000 7f8043200640 1 -- start start 2024-04-15T15:25:13.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8043200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f803c15eab0 0x7f803c162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8043200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f803c162b70 0x7f803c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8043200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 0x7f803c0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8043200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f803c0d2e50 con 0x7f803c15eab0 2024-04-15T15:25:13.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8043200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f803c0b78f0 con 0x7f803c162b70 2024-04-15T15:25:13.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8043200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f803c163010 con 0x7f803c0d1a90 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8041e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f803c15eab0 0x7f803c162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8041e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f803c15eab0 0x7f803c162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55292/0 (socket says 172.21.15.150:55292) 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.589+0000 7f8041e00640 1 -- 172.21.15.150:0/2996182157 learned_addr learned my addr 172.21.15.150:0/2996182157 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.590+0000 7f8041400640 1 --2- 172.21.15.150:0/2996182157 >> v2:172.21.15.187:3300/0 conn(0x7f803c162b70 0x7f803c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.590+0000 7f8042800640 1 --2- 172.21.15.150:0/2996182157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 0x7f803c0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.590+0000 7f8041e00640 1 -- 172.21.15.150:0/2996182157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 msgr2=0x7f803c0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.590+0000 7f8041e00640 1 --2- 172.21.15.150:0/2996182157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 0x7f803c0b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.590+0000 7f8041e00640 1 -- 172.21.15.150:0/2996182157 >> v2:172.21.15.187:3300/0 conn(0x7f803c162b70 msgr2=0x7f803c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.590+0000 7f8041e00640 1 --2- 172.21.15.150:0/2996182157 >> v2:172.21.15.187:3300/0 conn(0x7f803c162b70 0x7f803c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.590+0000 7f8041e00640 1 -- 172.21.15.150:0/2996182157 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c1632f0 con 0x7f803c15eab0 2024-04-15T15:25:13.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.590+0000 7f8042800640 1 --2- 172.21.15.150:0/2996182157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 0x7f803c0b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.591+0000 7f8041e00640 1 --2- 172.21.15.150:0/2996182157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c162630 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f803c0d5860 tx=0x7f803402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e42b077ff951d70 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.591+0000 7f8040a00640 1 -- 172.21.15.150:0/2996182157 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803402ca70 con 0x7f803c15eab0 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.591+0000 7f8040a00640 1 -- 172.21.15.150:0/2996182157 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803402cc10 con 0x7f803c15eab0 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.591+0000 7f8040a00640 1 -- 172.21.15.150:0/2996182157 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80340343f0 con 0x7f803c15eab0 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 -- 172.21.15.150:0/2996182157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 msgr2=0x7f803c162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 --2- 172.21.15.150:0/2996182157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c162630 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f803c0d5860 tx=0x7f803402bed0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 -- 172.21.15.150:0/2996182157 shutdown_connections 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 --2- 172.21.15.150:0/2996182157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 0x7f803c0b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 --2- 172.21.15.150:0/2996182157 >> v2:172.21.15.187:3300/0 conn(0x7f803c162b70 0x7f803c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 --2- 172.21.15.150:0/2996182157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c162630 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 -- 172.21.15.150:0/2996182157 >> 172.21.15.150:0/2996182157 conn(0x7f803c073240 msgr2=0x7f803c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:13.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 -- 172.21.15.150:0/2996182157 shutdown_connections 2024-04-15T15:25:13.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.592+0000 7f8043200640 1 -- 172.21.15.150:0/2996182157 wait complete. 2024-04-15T15:25:13.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.594+0000 7f8043200640 1 Processor -- start 2024-04-15T15:25:13.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.594+0000 7f8043200640 1 -- start start 2024-04-15T15:25:13.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8043200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 0x7f803c152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8043200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8043200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803c162b70 0x7f803c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8043200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0ba590 con 0x7f803c15eab0 2024-04-15T15:25:13.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8043200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f803c15ee90 con 0x7f803c162b70 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8043200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f803c0ba710 con 0x7f803c0d1a90 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8041400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8041400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55304/0 (socket says 172.21.15.150:55304) 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8041400640 1 -- 172.21.15.150:0/1109218519 learned_addr learned my addr 172.21.15.150:0/1109218519 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.595+0000 7f8042800640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803c162b70 0x7f803c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8041400640 1 -- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 msgr2=0x7f803c152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8041400640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 0x7f803c152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8041400640 1 -- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803c162b70 msgr2=0x7f803c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8041400640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803c162b70 0x7f803c0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8041400640 1 -- 172.21.15.150:0/1109218519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c1528c0 con 0x7f803c15eab0 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8042800640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803c162b70 0x7f803c0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:13.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8041400640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c154800 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d5f0 tx=0x7f802c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8032000640 1 -- 172.21.15.150:0/1109218519 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c0125f0 con 0x7f803c15eab0 2024-04-15T15:25:13.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8032000640 1 -- 172.21.15.150:0/1109218519 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802c019070 con 0x7f803c15eab0 2024-04-15T15:25:13.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8032000640 1 -- 172.21.15.150:0/1109218519 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c0129d0 con 0x7f803c15eab0 2024-04-15T15:25:13.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8043200640 1 -- 172.21.15.150:0/1109218519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c0cbe00 con 0x7f803c15eab0 2024-04-15T15:25:13.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.596+0000 7f8043200640 1 -- 172.21.15.150:0/1109218519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c0cc3c0 con 0x7f803c15eab0 2024-04-15T15:25:13.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.598+0000 7f8032000640 1 -- 172.21.15.150:0/1109218519 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f802c012b70 con 0x7f803c15eab0 2024-04-15T15:25:13.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.600+0000 7f8032000640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f802006c8f0 0x7f802006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.600+0000 7f8032000640 1 -- 172.21.15.150:0/1109218519 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f802c08bfb0 con 0x7f803c15eab0 2024-04-15T15:25:13.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.601+0000 7f8041e00640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f802006c8f0 0x7f802006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.601+0000 7f804758f740 1 Processor -- start 2024-04-15T15:25:13.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.601+0000 7f804758f740 1 -- start start 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f8041e00640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f802006c8f0 0x7f802006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8034035000 tx=0x7f803402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f804758f740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af251d4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f804758f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af2518e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f804758f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af2518ea00 0x55af2518c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f804758f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55af250f39c0 con 0x55af2518ea00 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f804758f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55af2509a560 con 0x55af251b5bc0 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f804758f740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55af250ef980 con 0x55af251b57e0 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f8042800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af2518ea00 0x55af2518c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f8042800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af2518ea00 0x55af2518c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55314/0 (socket says 172.21.15.150:55314) 2024-04-15T15:25:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.602+0000 7f8042800640 1 -- 172.21.15.150:0/3637388624 learned_addr learned my addr 172.21.15.150:0/3637388624 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8041400640 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af2518e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8041e00640 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af251d4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8042800640 1 -- 172.21.15.150:0/3637388624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 msgr2=0x55af251d4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8042800640 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af251d4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8042800640 1 -- 172.21.15.150:0/3637388624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 msgr2=0x55af2518e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8042800640 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af2518e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8042800640 1 -- 172.21.15.150:0/3637388624 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af24e062d0 con 0x55af2518ea00 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8042800640 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af2518ea00 0x55af2518c390 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f803800b970 tx=0x7f803800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae702a62d21fa9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8026600640 1 -- 172.21.15.150:0/3637388624 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038016070 con 0x55af2518ea00 2024-04-15T15:25:13.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8026600640 1 -- 172.21.15.150:0/3637388624 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803800dc10 con 0x55af2518ea00 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.603+0000 7f8026600640 1 -- 172.21.15.150:0/3637388624 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038012460 con 0x55af2518ea00 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 -- 172.21.15.150:0/3637388624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af2518ea00 msgr2=0x55af2518c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af2518ea00 0x55af2518c390 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f803800b970 tx=0x7f803800be30 comp rx=0 tx=0).stop 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 -- 172.21.15.150:0/3637388624 shutdown_connections 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af2518ea00 0x55af2518c390 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af2518e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 --2- 172.21.15.150:0/3637388624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af251d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 -- 172.21.15.150:0/3637388624 >> 172.21.15.150:0/3637388624 conn(0x55af2518aaa0 msgr2=0x55af251d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 -- 172.21.15.150:0/3637388624 shutdown_connections 2024-04-15T15:25:13.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.604+0000 7f804758f740 1 -- 172.21.15.150:0/3637388624 wait complete. 2024-04-15T15:25:13.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.605+0000 7f804758f740 1 -- start start 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.605+0000 7f804758f740 1 Processor -- start 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f804758f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b57e0 0x55af251e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f804758f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f804758f740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b6c60 0x55af251b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f804758f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55af251888d0 con 0x55af251b57e0 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f804758f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55af250ad920 con 0x55af251b5bc0 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f804758f740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55af2518ee80 con 0x55af251b6c60 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f804758f740 10 client.0 _ll_register_callbacks cb 0x55af250ad310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f8041e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b57e0 0x55af251e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.606+0000 7f8041e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b57e0 0x55af251e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55326/0 (socket says 172.21.15.150:55326) 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041e00640 1 -- 172.21.15.150:0/1352947668 learned_addr learned my addr 172.21.15.150:0/1352947668 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041400640 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041e00640 1 -- 172.21.15.150:0/1352947668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b6c60 msgr2=0x55af251b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041e00640 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b6c60 0x55af251b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041e00640 1 -- 172.21.15.150:0/1352947668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 msgr2=0x55af251ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041e00640 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041e00640 1 -- 172.21.15.150:0/1352947668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af24ed06b0 con 0x55af251b57e0 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041400640 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f8041e00640 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b57e0 0x55af251e6da0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f803402c4e0 tx=0x7f803402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d757f94dc40467ed server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f801aa00640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f801aa00640 1 -- 172.21.15.150:0/1352947668 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034051060 con 0x55af251b57e0 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f801aa00640 1 -- 172.21.15.150:0/1352947668 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8034040d20 con 0x55af251b57e0 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.607+0000 7f804758f740 1 -- 172.21.15.150:0/1352947668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803800dc10 con 0x55af251b57e0 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f801aa00640 1 -- 172.21.15.150:0/1352947668 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f803403e450 con 0x55af251b57e0 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f804758f740 10 client.5301 fetch_fsmap learned FSMap version 17 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f804758f740 1 -- 172.21.15.150:0/1352947668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55af251d0de0 con 0x55af251b57e0 2024-04-15T15:25:13.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f801aa00640 1 -- 172.21.15.150:0/1352947668 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803800dc10 con 0x55af251b57e0 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f801aa00640 1 -- 172.21.15.150:0/1352947668 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f8034008040 con 0x55af251b57e0 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f804758f740 10 client.5301 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f804758f740 10 client.5301 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f804758f740 20 client.5301 populate_metadata read hostname 'smithi150' 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f804758f740 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55af251d26f0 0x55af251f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f804758f740 4 client.5301 mds_command: new command op to 5112 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.608+0000 7f804758f740 1 -- 172.21.15.150:0/1352947668 --> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af25099b70 con 0x55af251d26f0 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.609+0000 7f8041400640 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55af251d26f0 0x55af251f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.609+0000 7f8041400640 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55af251d26f0 0x55af251f27f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d595bbb55c598e8a server_cookie=3862eb5aeb085671 in_seq=0 out_seq=0 2024-04-15T15:25:13.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.609+0000 7f801aa00640 10 client.5301 ms_handle_connect on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f801aa00640 1 -- 172.21.15.150:0/1352947668 <== mds.0 v2:172.21.15.150:6833/3327940880 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af25099b70 con 0x55af251d26f0 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f801aa00640 10 client.5301 handle_command_reply: tid=0 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 client.5301 shutdown 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 -- 172.21.15.150:0/1352947668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b57e0 msgr2=0x55af251e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b57e0 0x55af251e6da0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f803402c4e0 tx=0x7f803402e070 comp rx=0 tx=0).stop 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 -- 172.21.15.150:0/1352947668 shutdown_connections 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55af251d26f0 0x55af251f27f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b6c60 0x55af251b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 --2- 172.21.15.150:0/1352947668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b57e0 0x55af251e6da0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f801aa00640 0 client.5301 ms_handle_reset on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 -- 172.21.15.150:0/1352947668 >> 172.21.15.150:0/1352947668 conn(0x55af2518ea00 msgr2=0x55af2518d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:13.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 -- 172.21.15.150:0/1352947668 shutdown_connections 2024-04-15T15:25:13.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.610+0000 7f804758f740 1 -- 172.21.15.150:0/1352947668 wait complete. 2024-04-15T15:25:13.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.611+0000 7f804758f740 20 client.5301 trim_cache size 0 max 16384 2024-04-15T15:25:13.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.630+0000 7f804758f740 1 Processor -- start 2024-04-15T15:25:13.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.630+0000 7f804758f740 1 -- start start 2024-04-15T15:25:13.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f804758f740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af251b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f804758f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f804758f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af25188bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f804758f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f803c200c50 con 0x55af251b6c60 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f804758f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f803c161b00 con 0x55af251b5bc0 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f804758f740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f803c160e60 con 0x55af251b57e0 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f8042800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af25188bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f8042800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af25188bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55334/0 (socket says 172.21.15.150:55334) 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.631+0000 7f8042800640 1 -- 172.21.15.150:0/602369818 learned_addr learned my addr 172.21.15.150:0/602369818 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8041400640 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8042800640 1 -- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 msgr2=0x55af251b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8041e00640 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af251b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8042800640 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af251b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8042800640 1 -- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 msgr2=0x55af251886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8042800640 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8042800640 1 -- 172.21.15.150:0/602369818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af25187810 con 0x55af251b6c60 2024-04-15T15:25:13.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8041400640 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:13.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8042800640 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af25188bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8038015000 tx=0x7f8038010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b458b723075e9f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.632+0000 7f8025c00640 1 -- 172.21.15.150:0/602369818 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038016040 con 0x55af251b6c60 2024-04-15T15:25:13.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f8025c00640 1 -- 172.21.15.150:0/602369818 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803800b040 con 0x55af251b6c60 2024-04-15T15:25:13.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f8025c00640 1 -- 172.21.15.150:0/602369818 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800de10 con 0x55af251b6c60 2024-04-15T15:25:13.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 -- 172.21.15.150:0/602369818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 msgr2=0x55af25188bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af25188bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8038015000 tx=0x7f8038010000 comp rx=0 tx=0).stop 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 -- 172.21.15.150:0/602369818 shutdown_connections 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af25188bf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af251886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 --2- 172.21.15.150:0/602369818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af251b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 -- 172.21.15.150:0/602369818 >> 172.21.15.150:0/602369818 conn(0x55af2518ea00 msgr2=0x55af25183fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 -- 172.21.15.150:0/602369818 shutdown_connections 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.633+0000 7f804758f740 1 -- 172.21.15.150:0/602369818 wait complete. 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.634+0000 7f804758f740 1 -- start start 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.634+0000 7f804758f740 1 Processor -- start 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f804758f740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af2518d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f804758f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af2524d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f804758f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af251cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f804758f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55af251d1680 con 0x55af251b6c60 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f804758f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1614b0 con 0x55af251b5bc0 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f804758f740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55af251d2070 con 0x55af251b57e0 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f804758f740 10 client.0 _ll_register_callbacks cb 0x7f803402cce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f8042800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af251cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f8042800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af251cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55340/0 (socket says 172.21.15.150:55340) 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.635+0000 7f8042800640 1 -- 172.21.15.150:0/2407793621 learned_addr learned my addr 172.21.15.150:0/2407793621 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f8041400640 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af2524d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f8041e00640 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af2518d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f8042800640 1 -- 172.21.15.150:0/2407793621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 msgr2=0x55af2518d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f8042800640 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af2518d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f8042800640 1 -- 172.21.15.150:0/2407793621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 msgr2=0x55af2524d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f8042800640 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af2524d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f8042800640 1 -- 172.21.15.150:0/2407793621 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af251d6410 con 0x55af251b6c60 2024-04-15T15:25:13.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f8042800640 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af251cfe00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8038006b10 tx=0x7f8038004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ad5cf12eacdca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f801aa00640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:25:13.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.636+0000 7f801aa00640 1 -- 172.21.15.150:0/2407793621 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038035070 con 0x55af251b6c60 2024-04-15T15:25:13.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f801aa00640 1 -- 172.21.15.150:0/2407793621 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8038012810 con 0x55af251b6c60 2024-04-15T15:25:13.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 1 -- 172.21.15.150:0/2407793621 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803800b040 con 0x55af251b6c60 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f801aa00640 1 -- 172.21.15.150:0/2407793621 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8038012a10 con 0x55af251b6c60 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 10 client.5307 fetch_fsmap learned FSMap version 17 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 1 -- 172.21.15.150:0/2407793621 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55af251d7130 con 0x55af251b6c60 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f801aa00640 1 -- 172.21.15.150:0/2407793621 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803800b040 con 0x55af251b6c60 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f801aa00640 1 -- 172.21.15.150:0/2407793621 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f8038043080 con 0x55af251b6c60 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 10 client.5307 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 10 client.5307 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 20 client.5307 populate_metadata read hostname 'smithi150' 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55af251b4380 0x55af251dfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 4 client.5307 mds_command: new command op to 5112 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.637+0000 7f804758f740 1 -- 172.21.15.150:0/2407793621 --> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] -- command(tid 0: {"prefix": "status"}) -- 0x55af251cd6f0 con 0x55af251b4380 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.638+0000 7f8041e00640 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55af251b4380 0x55af251dfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.638+0000 7f8041e00640 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55af251b4380 0x55af251dfbf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22218e7509c1b45 server_cookie=2b5a6b114ced55ae in_seq=0 out_seq=0 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.638+0000 7f801aa00640 10 client.5307 ms_handle_connect on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.638+0000 7f801aa00640 1 -- 172.21.15.150:0/2407793621 <== mds.0 v2:172.21.15.150:6833/3327940880 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55af251cd6f0 con 0x55af251b4380 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.638+0000 7f801aa00640 10 client.5307 handle_command_reply: tid=0 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.638+0000 7f804758f740 1 client.5307 shutdown 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.638+0000 7f804758f740 1 -- 172.21.15.150:0/2407793621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 msgr2=0x55af251cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.638+0000 7f804758f740 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af251cfe00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8038006b10 tx=0x7f8038004730 comp rx=0 tx=0).stop 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 1 -- 172.21.15.150:0/2407793621 shutdown_connections 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55af251b4380 0x55af251dfbf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af251b6c60 0x55af251cfe00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af251b5bc0 0x55af2524d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 1 --2- 172.21.15.150:0/2407793621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55af251b57e0 0x55af2518d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 1 -- 172.21.15.150:0/2407793621 >> 172.21.15.150:0/2407793621 conn(0x55af2518ea00 msgr2=0x55af251e6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f801aa00640 0 client.5307 ms_handle_reset on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:13.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 1 -- 172.21.15.150:0/2407793621 shutdown_connections 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 1 -- 172.21.15.150:0/2407793621 wait complete. 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.639+0000 7f804758f740 20 client.5307 trim_cache size 0 max 16384 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4", 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 5112, 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 6.719504819, 2024-04-15T15:25:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 17, 2024-04-15T15:25:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 35, 2024-04-15T15:25:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 35, 2024-04-15T15:25:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 15.641821307000001 2024-04-15T15:25:13.646 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:25:13.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 -- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f802006c8f0 msgr2=0x7f802006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f802006c8f0 0x7f802006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8034035000 tx=0x7f803402e000 comp rx=0 tx=0).stop 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 -- 172.21.15.150:0/1109218519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 msgr2=0x7f803c154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c154800 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d5f0 tx=0x7f802c00dab0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 -- 172.21.15.150:0/1109218519 shutdown_connections 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f803c162b70 0x7f803c0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c15eab0 0x7f803c154800 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f802006c8f0 0x7f802006edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 --2- 172.21.15.150:0/1109218519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803c0d1a90 0x7f803c152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 -- 172.21.15.150:0/1109218519 >> 172.21.15.150:0/1109218519 conn(0x7f803c073240 msgr2=0x7f803c0c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 -- 172.21.15.150:0/1109218519 shutdown_connections 2024-04-15T15:25:13.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.640+0000 7f8043200640 1 -- 172.21.15.150:0/1109218519 wait complete. 2024-04-15T15:25:13.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:25:13.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.864+0000 7fd3e1a00640 1 Processor -- start 2024-04-15T15:25:13.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.864+0000 7fd3e1a00640 1 -- start start 2024-04-15T15:25:13.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3e1a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd3dc1461f0 0x7fd3dc0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3e1a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0cedf0 0x7fd3dc0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3e1a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3dc0b4be0 con 0x7fd3dc0b3e10 2024-04-15T15:25:13.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3e1a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd3dc0a9640 con 0x7fd3dc1461f0 2024-04-15T15:25:13.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3e1a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3dc0cf250 con 0x7fd3dc0cedf0 2024-04-15T15:25:13.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3db400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd3dc0b3e10 0x7fd3dc0cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:55218/0 (socket says 172.21.15.150:55218) 2024-04-15T15:25:13.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3db400640 1 -- 172.21.15.150:0/1440983460 learned_addr learned my addr 172.21.15.150:0/1440983460 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:13.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3daa00640 1 --2- 172.21.15.150:0/1440983460 >> v2:172.21.15.187:3300/0 conn(0x7fd3dc1461f0 0x7fd3dc0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.866+0000 7fd3da000640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0cedf0 0x7fd3dc0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1863477440 0 0) 0x7fd3dc0b4be0 con 0x7fd3dc0b3e10 2024-04-15T15:25:13.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3c0003520 con 0x7fd3dc0b3e10 2024-04-15T15:25:13.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1835664773 0 0) 0x7fd3c0003520 con 0x7fd3dc0b3e10 2024-04-15T15:25:13.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0cedf0 msgr2=0x7fd3dc0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0cedf0 0x7fd3dc0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 >> v2:172.21.15.187:3300/0 conn(0x7fd3dc1461f0 msgr2=0x7fd3dc0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 --2- 172.21.15.150:0/1440983460 >> v2:172.21.15.187:3300/0 conn(0x7fd3dc1461f0 0x7fd3dc0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc0b4620 con 0x7fd3dc0b3e10 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fd3d40019f0 con 0x7fd3dc0b3e10 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 >> v1:172.21.15.150:6789/0 conn(0x7fd3dc0b3e10 legacy=0x7fd3dc0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0cedf0 0x7fd3c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c0003f30 0x7fd3c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0004d40 0x7fd3c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c000b9c0 con 0x7fd3c0003f30 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c000bb40 con 0x7fd3c0004d40 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.867+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c000bcc0 con 0x7fd3dc0cedf0 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.868+0000 7fd3da000640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c0003f30 0x7fd3c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.868+0000 7fd3db400640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0004d40 0x7fd3c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.868+0000 7fd3da000640 1 -- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0cedf0 msgr2=0x7fd3c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.868+0000 7fd3da000640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0cedf0 0x7fd3c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.868+0000 7fd3da000640 1 -- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0004d40 msgr2=0x7fd3c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:13.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.868+0000 7fd3da000640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0004d40 0x7fd3c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.868+0000 7fd3da000640 1 -- 172.21.15.150:0/1440983460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c0003520 con 0x7fd3c0003f30 2024-04-15T15:25:13.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.868+0000 7fd3db400640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0004d40 0x7fd3c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:13.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.869+0000 7fd3da000640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c0003f30 0x7fd3c0004800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009ac0 tx=0x7fd3c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeae60beb9ec32ef server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.869+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8039070 con 0x7fd3c0003f30 2024-04-15T15:25:13.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.869+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c802c6d0 con 0x7fd3c0003f30 2024-04-15T15:25:13.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.869+0000 7fd3d9600640 1 -- 172.21.15.150:0/1440983460 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8009690 con 0x7fd3c0003f30 2024-04-15T15:25:13.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.870+0000 7fd3e1a00640 1 -- 172.21.15.150:0/1440983460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c0003f30 msgr2=0x7fd3c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.870+0000 7fd3e1a00640 1 --2- 172.21.15.150:0/1440983460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3c0003f30 0x7fd3c0004800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009ac0 tx=0x7fd3c0005830 comp rx=0 tx=0).stop 2024-04-15T15:25:13.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.870+0000 7fd3db400640 1 -- 172.21.15.150:0/1440983460 reap_dead start 2024-04-15T15:25:13.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.870+0000 7fd3e1a00640 1 -- 172.21.15.150:0/1440983460 shutdown_connections 2024-04-15T15:25:13.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.870+0000 7fd3e1a00640 1 -- 172.21.15.150:0/1440983460 >> 172.21.15.150:0/1440983460 conn(0x7fd3dc0b27b0 msgr2=0x7fd3dc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:13.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.870+0000 7fd3e1a00640 1 -- 172.21.15.150:0/1440983460 shutdown_connections 2024-04-15T15:25:13.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.870+0000 7fd3e1a00640 1 -- 172.21.15.150:0/1440983460 wait complete. 2024-04-15T15:25:13.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.872+0000 7fd3e1a00640 1 Processor -- start 2024-04-15T15:25:13.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.872+0000 7fd3e1a00640 1 -- start start 2024-04-15T15:25:13.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3e1a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0003f30 0x7fd3dc1433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3e1a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3dc0c8630 0x7fd3dc15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3e1a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0c8ab0 0x7fd3dc15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3e1a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc15d290 con 0x7fd3dc0c8630 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3e1a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc15cfb0 con 0x7fd3c0003f30 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3e1a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3dc1465d0 con 0x7fd3dc0c8ab0 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3da000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3dc0c8630 0x7fd3dc15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3da000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3dc0c8630 0x7fd3dc15d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55358/0 (socket says 172.21.15.150:55358) 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.873+0000 7fd3da000640 1 -- 172.21.15.150:0/556961118 learned_addr learned my addr 172.21.15.150:0/556961118 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.874+0000 7fd3daa00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0003f30 0x7fd3dc1433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.874+0000 7fd3db400640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0c8ab0 0x7fd3dc15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.874+0000 7fd3da000640 1 -- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0c8ab0 msgr2=0x7fd3dc15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.874+0000 7fd3da000640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0c8ab0 0x7fd3dc15dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.874+0000 7fd3da000640 1 -- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0003f30 msgr2=0x7fd3dc1433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.874+0000 7fd3da000640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0003f30 0x7fd3dc1433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:13.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.874+0000 7fd3da000640 1 -- 172.21.15.150:0/556961118 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc0c8ff0 con 0x7fd3dc0c8630 2024-04-15T15:25:13.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.874+0000 7fd3da000640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3dc0c8630 0x7fd3dc15d6b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000c00 tx=0x7fd3c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.875+0000 7fd3d2a00640 1 -- 172.21.15.150:0/556961118 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8042020 con 0x7fd3dc0c8630 2024-04-15T15:25:13.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.875+0000 7fd3d2a00640 1 -- 172.21.15.150:0/556961118 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c803a040 con 0x7fd3dc0c8630 2024-04-15T15:25:13.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.875+0000 7fd3d2a00640 1 -- 172.21.15.150:0/556961118 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8039070 con 0x7fd3dc0c8630 2024-04-15T15:25:13.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.875+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc0c5a90 con 0x7fd3dc0c8630 2024-04-15T15:25:13.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.875+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc0c6050 con 0x7fd3dc0c8630 2024-04-15T15:25:13.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.876+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc068c80 con 0x7fd3dc0c8630 2024-04-15T15:25:13.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.877+0000 7fd3d2a00640 1 -- 172.21.15.150:0/556961118 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3c8045020 con 0x7fd3dc0c8630 2024-04-15T15:25:13.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.878+0000 7fd3d2a00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd3bc06c920 0x7fd3bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:13.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.878+0000 7fd3d2a00640 1 -- 172.21.15.150:0/556961118 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd3c80acf30 con 0x7fd3dc0c8630 2024-04-15T15:25:13.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.878+0000 7fd3daa00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd3bc06c920 0x7fd3bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:13.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.879+0000 7fd3daa00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd3bc06c920 0x7fd3bc06ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0b1620 tx=0x7fd3cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:13.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:13.882+0000 7fd3d2a00640 1 -- 172.21.15.150:0/556961118 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3c804c070 con 0x7fd3dc0c8630 2024-04-15T15:25:14.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.125+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd3dc0b1ef0 con 0x7fd3dc0c8630 2024-04-15T15:25:14.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.126+0000 7fd3d2a00640 1 -- 172.21.15.150:0/556961118 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fd3c807b350 con 0x7fd3dc0c8630 2024-04-15T15:25:14.128 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:25:14.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.128+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd3bc06c920 msgr2=0x7fd3bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.128+0000 7fd3e1a00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd3bc06c920 0x7fd3bc06ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0b1620 tx=0x7fd3cc008040 comp rx=0 tx=0).stop 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.128+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3dc0c8630 msgr2=0x7fd3dc15d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.128+0000 7fd3e1a00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3dc0c8630 0x7fd3dc15d6b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000c00 tx=0x7fd3c8004730 comp rx=0 tx=0).stop 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.129+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 shutdown_connections 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.129+0000 7fd3e1a00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fd3bc06c920 0x7fd3bc06ede0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.129+0000 7fd3e1a00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3dc0c8ab0 0x7fd3dc15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.129+0000 7fd3e1a00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3dc0c8630 0x7fd3dc15d6b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.129+0000 7fd3e1a00640 1 --2- 172.21.15.150:0/556961118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3c0003f30 0x7fd3dc1433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.129+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 >> 172.21.15.150:0/556961118 conn(0x7fd3dc0b27b0 msgr2=0x7fd3dc0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.129+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 shutdown_connections 2024-04-15T15:25:14.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14.129+0000 7fd3e1a00640 1 -- 172.21.15.150:0/556961118 wait complete. 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:25:14.150 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:25:14.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:14.151 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:14.199 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42501sec preferred_lft 42501sec 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 3e:e8:cf:d9:d1:9d brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::1086:faff:fe4c:731a/64 scope link 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:14.200 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 76:7b:44:2c:2d:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:25:14.201 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::747b:44ff:fe2c:2d50/64 scope link 2024-04-15T15:25:14.201 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:14.201 INFO:teuthology.orchestra.run.smithi150.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:14.201 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 3e:e8:cf:d9:d1:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:25:14.201 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::3ce8:cfff:fed9:d19d/64 scope link 2024-04-15T15:25:14.201 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:14.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:14.202 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:25:14.286 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-15T15:25:14.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:14.290 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-04-15T15:25:14.340 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-04-15T15:25:14.340 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:25:14.340 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:25:14.340 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:25:14.340 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:25:14.341 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:25:14.341 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:25:14.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:25:14.342 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:25:14.342 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:25:14.342 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:25:14.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:25:14.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:14.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:14.581 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:25:14.629 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:25:14.629 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:25:14.630 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:25:14.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:14.630 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-04-15T15:25:14.721 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:14.721 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:14.774 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:25:14.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:14.775 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:14.823 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:25:14.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:14.823 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:25:14.967 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:25:14.967 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:25:14.967 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:14.987 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:25:14.987 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:25:14.987 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:14.988 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:14.988 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:15.013 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:25:15.013 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:25:15.013 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:15.013 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:15.013 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:15.014 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:25:15.014 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-04-15T15:25:15.014 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:25:15.016 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-15T15:25:15.016 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:25:15.016 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:25:15.016 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:25:15.016 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:25:15.020 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:25:15.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:15.020 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:15.098 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:25:15.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:25:15.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:15.234 INFO:teuthology.orchestra.run.smithi150.stdout:client4825 2024-04-15T15:25:15.235 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:25:15.235 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:15.288 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:25:15.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.499+0000 7f4db6e00640 1 Processor -- start 2024-04-15T15:25:15.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.499+0000 7f4db6e00640 1 -- start start 2024-04-15T15:25:15.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.501+0000 7f4db6e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4db00cf110 0x7f4db0162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:15.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.501+0000 7f4db6e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db00d0090 con 0x7f4db0162020 2024-04-15T15:25:15.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.501+0000 7f4db6e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db00cfad0 con 0x7f4db00d1c90 2024-04-15T15:25:15.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.501+0000 7f4db6e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4db0163440 con 0x7f4db00cf110 2024-04-15T15:25:15.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.501+0000 7f4db5a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f4db0162020 0x7f4db00d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:55224/0 (socket says 172.21.15.150:55224) 2024-04-15T15:25:15.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4db5a00640 1 -- 172.21.15.150:0/2021426508 learned_addr learned my addr 172.21.15.150:0/2021426508 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:15.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4db6400640 1 --2- 172.21.15.150:0/2021426508 >> v2:172.21.15.187:3300/0 conn(0x7f4db00cf110 0x7f4db0162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:15.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541608385 0 0) 0x7f4da4001110 con 0x7f4db0162020 2024-04-15T15:25:15.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d98003520 con 0x7f4db0162020 2024-04-15T15:25:15.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3316561947 0 0) 0x7f4d98003520 con 0x7f4db0162020 2024-04-15T15:25:15.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 >> v2:172.21.15.187:3300/0 conn(0x7f4db00cf110 msgr2=0x7f4db0162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:15.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 --2- 172.21.15.150:0/2021426508 >> v2:172.21.15.187:3300/0 conn(0x7f4db00cf110 0x7f4db0162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 >> v1:172.21.15.187:6789/0 conn(0x7f4db00d1c90 legacy=0x7f4db00d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:15.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db00cfdb0 con 0x7f4db0162020 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f4da4001840 con 0x7f4db0162020 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.502+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 >> v1:172.21.15.150:6789/0 conn(0x7f4db0162020 legacy=0x7f4db00d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4dafe00640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d98003a00 0x7f4d98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4dafe00640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d98004800 0x7f4d98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4dafe00640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d980051c0 0x7f4d9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0163440 con 0x7f4d980051c0 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9800c0c0 con 0x7f4d98003a00 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9800c2c0 con 0x7f4d98004800 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4db6400640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d980051c0 0x7f4d9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4db6400640 1 -- 172.21.15.150:0/2021426508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d98004800 msgr2=0x7f4d98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4db6400640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d98004800 0x7f4d98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4db6400640 1 -- 172.21.15.150:0/2021426508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d98003a00 msgr2=0x7f4d98003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:15.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4db5a00640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d98003a00 0x7f4d98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:15.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4db6400640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d98003a00 0x7f4d98003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4db6400640 1 -- 172.21.15.150:0/2021426508 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d98003520 con 0x7f4d980051c0 2024-04-15T15:25:15.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.503+0000 7f4db5a00640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d98003a00 0x7f4d98003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:15.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.504+0000 7f4db6400640 1 -- 172.21.15.150:0/2021426508 reap_dead start 2024-04-15T15:25:15.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4db6400640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d980051c0 0x7f4d9800b9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4db00cf560 tx=0x7f4d98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc101d533be4b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:15.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8009c10 con 0x7f4d980051c0 2024-04-15T15:25:15.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da802cbd0 con 0x7f4d980051c0 2024-04-15T15:25:15.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4dafe00640 1 -- 172.21.15.150:0/2021426508 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8003990 con 0x7f4d980051c0 2024-04-15T15:25:15.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4db6e00640 1 -- 172.21.15.150:0/2021426508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d980051c0 msgr2=0x7f4d9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:15.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4db6e00640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d980051c0 0x7f4d9800b9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4db00cf560 tx=0x7f4d98005830 comp rx=0 tx=0).stop 2024-04-15T15:25:15.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4db6e00640 1 -- 172.21.15.150:0/2021426508 shutdown_connections 2024-04-15T15:25:15.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4db6e00640 1 --2- 172.21.15.150:0/2021426508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d980051c0 0x7f4d9800b9c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.505+0000 7f4db6e00640 1 -- 172.21.15.150:0/2021426508 >> 172.21.15.150:0/2021426508 conn(0x7f4db006cf00 msgr2=0x7f4db006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:15.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.506+0000 7f4db6e00640 1 -- 172.21.15.150:0/2021426508 shutdown_connections 2024-04-15T15:25:15.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.506+0000 7f4db6e00640 1 -- 172.21.15.150:0/2021426508 wait complete. 2024-04-15T15:25:15.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.507+0000 7f4db6e00640 1 Processor -- start 2024-04-15T15:25:15.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.508+0000 7f4db6e00640 1 -- start start 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d980051c0 0x7f4db014f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4db00c8350 0x7f4db0160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db015a8c0 0x7f4db0161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4db00cf880 con 0x7f4db015a8c0 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4db00b7b30 con 0x7f4db00c8350 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4db00d2e70 con 0x7f4d980051c0 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db015a8c0 0x7f4db0161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db015a8c0 0x7f4db0161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55376/0 (socket says 172.21.15.150:55376) 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6400640 1 -- 172.21.15.150:0/2919955579 learned_addr learned my addr 172.21.15.150:0/2919955579 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6400640 1 -- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d980051c0 msgr2=0x7f4db014f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:15.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6400640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d980051c0 0x7f4db014f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6400640 1 -- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4db00c8350 msgr2=0x7f4db0160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:15.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db5000640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4db00c8350 0x7f4db0160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:15.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6400640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4db00c8350 0x7f4db0160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.509+0000 7f4db6400640 1 -- 172.21.15.150:0/2919955579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db00c87e0 con 0x7f4db015a8c0 2024-04-15T15:25:15.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.510+0000 7f4db6400640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db015a8c0 0x7f4db0161120 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4da800e200 tx=0x7f4da8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:15.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.510+0000 7f4dad600640 1 -- 172.21.15.150:0/2919955579 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8009c10 con 0x7f4db015a8c0 2024-04-15T15:25:15.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.510+0000 7f4dad600640 1 -- 172.21.15.150:0/2919955579 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da8044070 con 0x7f4db015a8c0 2024-04-15T15:25:15.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.510+0000 7f4dad600640 1 -- 172.21.15.150:0/2919955579 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8004860 con 0x7f4db015a8c0 2024-04-15T15:25:15.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.510+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db00c8ac0 con 0x7f4db015a8c0 2024-04-15T15:25:15.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.510+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db00c9080 con 0x7f4db015a8c0 2024-04-15T15:25:15.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.511+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db00755c0 con 0x7f4db015a8c0 2024-04-15T15:25:15.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.512+0000 7f4dad600640 1 -- 172.21.15.150:0/2919955579 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4da8005d10 con 0x7f4db015a8c0 2024-04-15T15:25:15.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.513+0000 7f4dad600640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4d9006c8f0 0x7f4d9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:15.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.513+0000 7f4dad600640 1 -- 172.21.15.150:0/2919955579 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f4da80ae300 con 0x7f4db015a8c0 2024-04-15T15:25:15.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.514+0000 7f4db5a00640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4d9006c8f0 0x7f4d9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:15.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.515+0000 7f4db5a00640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4d9006c8f0 0x7f4d9006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4da4001eb0 tx=0x7f4da4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:15.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.518+0000 7f4dad600640 1 -- 172.21.15.150:0/2919955579 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4da807c690 con 0x7f4db015a8c0 2024-04-15T15:25:15.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.757+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4db00b7650 con 0x7f4db015a8c0 2024-04-15T15:25:15.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.758+0000 7f4dad600640 1 -- 172.21.15.150:0/2919955579 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f4da807bf90 con 0x7f4db015a8c0 2024-04-15T15:25:15.760 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:25:15.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.760+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4d9006c8f0 msgr2=0x7f4d9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:15.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.760+0000 7f4db6e00640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4d9006c8f0 0x7f4d9006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4da4001eb0 tx=0x7f4da4009040 comp rx=0 tx=0).stop 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.760+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db015a8c0 msgr2=0x7f4db0161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.760+0000 7f4db6e00640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db015a8c0 0x7f4db0161120 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4da800e200 tx=0x7f4da8009750 comp rx=0 tx=0).stop 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.761+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 shutdown_connections 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.761+0000 7f4db6e00640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db015a8c0 0x7f4db0161120 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.761+0000 7f4db6e00640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4db00c8350 0x7f4db0160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.761+0000 7f4db6e00640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4d9006c8f0 0x7f4d9006edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.761+0000 7f4db6e00640 1 --2- 172.21.15.150:0/2919955579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d980051c0 0x7f4db014f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.761+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 >> 172.21.15.150:0/2919955579 conn(0x7f4db006cf00 msgr2=0x7f4db0157f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.761+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 shutdown_connections 2024-04-15T15:25:15.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:15.761+0000 7f4db6e00640 1 -- 172.21.15.150:0/2919955579 wait complete. 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:25:15.780 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:25:15.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:15.781 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-04-15T15:25:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:25:15.839 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:25:15.839 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:15.840 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42547sec preferred_lft 42547sec 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 16:44:cc:5d:db:12 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::442c:2bff:fe33:cf6c/64 scope link 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 5a:c2:59:f5:7f:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::58c2:59ff:fef5:7f4e/64 scope link 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:15.842 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 16:44:cc:5d:db:12 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:25:15.842 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::1444:ccff:fe5d:db12/64 scope link 2024-04-15T15:25:15.842 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:15.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:15.843 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:25:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-15T15:25:15.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:15.937 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-04-15T15:25:15.989 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-04-15T15:25:15.989 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:25:15.989 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:25:15.989 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:25:15.989 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:25:15.990 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:25:15.990 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:25:15.990 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:25:15.990 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:25:15.990 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:25:15.990 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:25:16.095 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:25:16.225 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:16.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:16.234 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:25:16.279 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:25:16.279 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:25:16.279 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:25:16.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:16.280 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-04-15T15:25:16.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:16.415 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:16.471 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:25:16.471 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:16.471 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:16.524 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-15T15:25:16.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:16.524 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:25:16.660 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:25:16.660 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:25:16.660 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:16.682 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:25:16.682 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:25:16.682 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:16.682 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:16.682 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:16.710 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:25:16.710 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:25:16.710 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:16.710 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:16.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:16.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:25:16.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-04-15T15:25:16.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:25:16.713 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-15T15:25:16.713 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:25:16.713 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:25:16.713 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:25:16.713 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:25:16.719 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:25:16.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:16.719 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:16.798 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:25:16.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:25:16.833 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:25:16.899 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:25:16.903 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:16.912 INFO:teuthology.orchestra.run.smithi187.stdout:client4628 2024-04-15T15:25:16.912 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:25:16.913 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:16.954 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 25 2024-04-15T15:25:17.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.174+0000 7f0a45c00640 1 Processor -- start 2024-04-15T15:25:17.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.174+0000 7f0a45c00640 1 -- start start 2024-04-15T15:25:17.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.176+0000 7f0a45c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a40158f10 0x7f0a4006f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.176+0000 7f0a45c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0a4006f570 0x7f0a400cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.176+0000 7f0a45c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a4015f230 con 0x7f0a400b3dc0 2024-04-15T15:25:17.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.176+0000 7f0a45c00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0a4006f9c0 con 0x7f0a4006f570 2024-04-15T15:25:17.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.176+0000 7f0a45c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4015f4d0 con 0x7f0a40158f10 2024-04-15T15:25:17.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.176+0000 7f0a45200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0a4006f570 0x7f0a400cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.177+0000 7f0a45200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0a4006f570 0x7f0a400cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55954/0 (socket says 172.21.15.150:55954) 2024-04-15T15:25:17.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.177+0000 7f0a45200640 1 -- 172.21.15.150:0/4268746637 learned_addr learned my addr 172.21.15.150:0/4268746637 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:17.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.177+0000 7f0a3fe00640 1 --2- 172.21.15.150:0/4268746637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a40158f10 0x7f0a4006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.177+0000 7f0a45200640 1 -- 172.21.15.150:0/4268746637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a40158f10 msgr2=0x7f0a4006f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:17.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.177+0000 7f0a45200640 1 --2- 172.21.15.150:0/4268746637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a40158f10 0x7f0a4006f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.177+0000 7f0a45200640 1 -- 172.21.15.150:0/4268746637 >> v1:172.21.15.187:6789/0 conn(0x7f0a400b3dc0 legacy=0x7f0a400cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:17.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.177+0000 7f0a45200640 1 -- 172.21.15.150:0/4268746637 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4015f7b0 con 0x7f0a4006f570 2024-04-15T15:25:17.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.177+0000 7f0a3fe00640 1 --2- 172.21.15.150:0/4268746637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a40158f10 0x7f0a4006f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:17.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.178+0000 7f0a45200640 1 --2- 172.21.15.150:0/4268746637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4006f570 0x7f0a400cdbf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0a400702f0 tx=0x7f0a3802e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949223aa3a80f2ba server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:17.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.178+0000 7f0a3f400640 1 -- 172.21.15.150:0/4268746637 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3802ebd0 con 0x7f0a4006f570 2024-04-15T15:25:17.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.178+0000 7f0a3f400640 1 -- 172.21.15.150:0/4268746637 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a3802ed70 con 0x7f0a4006f570 2024-04-15T15:25:17.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.178+0000 7f0a3f400640 1 -- 172.21.15.150:0/4268746637 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38036610 con 0x7f0a4006f570 2024-04-15T15:25:17.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.179+0000 7f0a45c00640 1 -- 172.21.15.150:0/4268746637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4006f570 msgr2=0x7f0a400cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.179+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4268746637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4006f570 0x7f0a400cdbf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0a400702f0 tx=0x7f0a3802e030 comp rx=0 tx=0).stop 2024-04-15T15:25:17.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.179+0000 7f0a45c00640 1 -- 172.21.15.150:0/4268746637 shutdown_connections 2024-04-15T15:25:17.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.179+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4268746637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4006f570 0x7f0a400cdbf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.179+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4268746637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a40158f10 0x7f0a4006f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.179+0000 7f0a45c00640 1 -- 172.21.15.150:0/4268746637 >> 172.21.15.150:0/4268746637 conn(0x7f0a4006b930 msgr2=0x7f0a4006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:17.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.179+0000 7f0a45c00640 1 -- 172.21.15.150:0/4268746637 shutdown_connections 2024-04-15T15:25:17.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.179+0000 7f0a45c00640 1 -- 172.21.15.150:0/4268746637 wait complete. 2024-04-15T15:25:17.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.181+0000 7f0a45c00640 1 Processor -- start 2024-04-15T15:25:17.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.181+0000 7f0a45c00640 1 -- start start 2024-04-15T15:25:17.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.182+0000 7f0a45c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a4006f570 0x7f0a40153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.182+0000 7f0a45c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a400b3dc0 0x7f0a4014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.182+0000 7f0a45c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a40158f10 0x7f0a4014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.182+0000 7f0a45c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0a401592f0 con 0x7f0a40158f10 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.182+0000 7f0a45c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4015fd70 con 0x7f0a4006f570 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.182+0000 7f0a45c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0a40159470 con 0x7f0a400b3dc0 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a40158f10 0x7f0a4014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a40158f10 0x7f0a4014da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55962/0 (socket says 172.21.15.150:55962) 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 -- 172.21.15.150:0/4192072087 learned_addr learned my addr 172.21.15.150:0/4192072087 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a3fe00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a4006f570 0x7f0a40153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a37e00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a400b3dc0 0x7f0a4014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 -- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a400b3dc0 msgr2=0x7f0a4014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a400b3dc0 0x7f0a4014d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 -- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a4006f570 msgr2=0x7f0a40153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a4006f570 0x7f0a40153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 -- 172.21.15.150:0/4192072087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4015a5a0 con 0x7f0a40158f10 2024-04-15T15:25:17.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a37e00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a400b3dc0 0x7f0a4014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:17.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a3fe00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a4006f570 0x7f0a40153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:17.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.183+0000 7f0a45200640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a40158f10 0x7f0a4014da50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0a38038040 tx=0x7f0a38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:17.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.184+0000 7f0a3cc00640 1 -- 172.21.15.150:0/4192072087 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3802e9d0 con 0x7f0a40158f10 2024-04-15T15:25:17.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.184+0000 7f0a3cc00640 1 -- 172.21.15.150:0/4192072087 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a38044070 con 0x7f0a40158f10 2024-04-15T15:25:17.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.184+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4015a880 con 0x7f0a40158f10 2024-04-15T15:25:17.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.184+0000 7f0a3cc00640 1 -- 172.21.15.150:0/4192072087 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3803f470 con 0x7f0a40158f10 2024-04-15T15:25:17.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.184+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4015ae40 con 0x7f0a40158f10 2024-04-15T15:25:17.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.185+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a40156c70 con 0x7f0a40158f10 2024-04-15T15:25:17.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.186+0000 7f0a3cc00640 1 -- 172.21.15.150:0/4192072087 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a3803f610 con 0x7f0a40158f10 2024-04-15T15:25:17.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.187+0000 7f0a3cc00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0a1c06c920 0x7f0a1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.187+0000 7f0a3cc00640 1 -- 172.21.15.150:0/4192072087 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0a380ad980 con 0x7f0a40158f10 2024-04-15T15:25:17.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.188+0000 7f0a3fe00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0a1c06c920 0x7f0a1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.189+0000 7f0a3fe00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0a1c06c920 0x7f0a1c06ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a30006fd0 tx=0x7f0a30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:17.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.191+0000 7f0a3cc00640 1 -- 172.21.15.150:0/4192072087 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a3804d020 con 0x7f0a40158f10 2024-04-15T15:25:17.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.393+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f0a400cc680 con 0x7f0a40158f10 2024-04-15T15:25:17.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.400+0000 7f0a3cc00640 1 -- 172.21.15.150:0/4192072087 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v11)=0 v11) ==== 108+0+0 (secure 0 0 0) 0x7f0a3807bcc0 con 0x7f0a40158f10 2024-04-15T15:25:17.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0a1c06c920 msgr2=0x7f0a1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0a1c06c920 0x7f0a1c06ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a30006fd0 tx=0x7f0a30008040 comp rx=0 tx=0).stop 2024-04-15T15:25:17.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a40158f10 msgr2=0x7f0a4014da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a40158f10 0x7f0a4014da50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0a38038040 tx=0x7f0a38004a40 comp rx=0 tx=0).stop 2024-04-15T15:25:17.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 shutdown_connections 2024-04-15T15:25:17.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a40158f10 0x7f0a4014da50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f0a1c06c920 0x7f0a1c06ede0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a400b3dc0 0x7f0a4014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 --2- 172.21.15.150:0/4192072087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a4006f570 0x7f0a40153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 >> 172.21.15.150:0/4192072087 conn(0x7f0a4006b930 msgr2=0x7f0a400cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:17.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.403+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 shutdown_connections 2024-04-15T15:25:17.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.404+0000 7f0a45c00640 1 -- 172.21.15.150:0/4192072087 wait complete. 2024-04-15T15:25:17.425 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_quiesce_splitauth true 2024-04-15T15:25:17.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.636+0000 7f93bba00640 1 Processor -- start 2024-04-15T15:25:17.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.636+0000 7f93bba00640 1 -- start start 2024-04-15T15:25:17.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.638+0000 7f93bba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b4145170 0x7f93b40cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.638+0000 7f93bba00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f93b40ce070 0x7f93b40b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.638+0000 7f93bba00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b40b4410 con 0x7f93b40b59f0 2024-04-15T15:25:17.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.638+0000 7f93bba00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f93b4145550 con 0x7f93b40ce070 2024-04-15T15:25:17.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.638+0000 7f93bba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93b40b46f0 con 0x7f93b4145170 2024-04-15T15:25:17.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.638+0000 7f93b3e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f93b40ce070 0x7f93b40b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.638+0000 7f93b3e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f93b40ce070 0x7f93b40b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55966/0 (socket says 172.21.15.150:55966) 2024-04-15T15:25:17.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.638+0000 7f93b3e00640 1 -- 172.21.15.150:0/60394204 learned_addr learned my addr 172.21.15.150:0/60394204 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:17.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.639+0000 7f93b9000640 1 --2- 172.21.15.150:0/60394204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b4145170 0x7f93b40cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.639+0000 7f93b3e00640 1 -- 172.21.15.150:0/60394204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b4145170 msgr2=0x7f93b40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:17.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.639+0000 7f93b3e00640 1 --2- 172.21.15.150:0/60394204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b4145170 0x7f93b40cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.639+0000 7f93b3e00640 1 -- 172.21.15.150:0/60394204 >> v1:172.21.15.187:6789/0 conn(0x7f93b40b59f0 legacy=0x7f93b406f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:17.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.639+0000 7f93b3e00640 1 -- 172.21.15.150:0/60394204 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b40b49d0 con 0x7f93b40ce070 2024-04-15T15:25:17.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.640+0000 7f93b3e00640 1 --2- 172.21.15.150:0/60394204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40ce070 0x7f93b40b54b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f939c009870 tx=0x7f939c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4ad7a1effd0822 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:17.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.640+0000 7f93b3400640 1 -- 172.21.15.150:0/60394204 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c035020 con 0x7f93b40ce070 2024-04-15T15:25:17.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.640+0000 7f93b3400640 1 -- 172.21.15.150:0/60394204 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939c031640 con 0x7f93b40ce070 2024-04-15T15:25:17.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.640+0000 7f93b3400640 1 -- 172.21.15.150:0/60394204 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c02d040 con 0x7f93b40ce070 2024-04-15T15:25:17.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.641+0000 7f93bba00640 1 -- 172.21.15.150:0/60394204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40ce070 msgr2=0x7f93b40b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.641+0000 7f93bba00640 1 --2- 172.21.15.150:0/60394204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40ce070 0x7f93b40b54b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f939c009870 tx=0x7f939c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.641+0000 7f93bba00640 1 -- 172.21.15.150:0/60394204 shutdown_connections 2024-04-15T15:25:17.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.641+0000 7f93bba00640 1 --2- 172.21.15.150:0/60394204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40ce070 0x7f93b40b54b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.641+0000 7f93bba00640 1 --2- 172.21.15.150:0/60394204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b4145170 0x7f93b40cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.641+0000 7f93bba00640 1 -- 172.21.15.150:0/60394204 >> 172.21.15.150:0/60394204 conn(0x7f93b40b2460 msgr2=0x7f93b40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:17.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.641+0000 7f93bba00640 1 -- 172.21.15.150:0/60394204 shutdown_connections 2024-04-15T15:25:17.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.641+0000 7f93bba00640 1 -- 172.21.15.150:0/60394204 wait complete. 2024-04-15T15:25:17.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.643+0000 7f93bba00640 1 Processor -- start 2024-04-15T15:25:17.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.644+0000 7f93bba00640 1 -- start start 2024-04-15T15:25:17.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93bba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b40b59f0 0x7f93b4141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93bba00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b40ce070 0x7f93b4141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93bba00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4145170 0x7f93b40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93bba00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40b4030 con 0x7f93b4145170 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93bba00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40b4f90 con 0x7f93b40ce070 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93bba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93b40b41b0 con 0x7f93b40b59f0 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93b9a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4145170 0x7f93b40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93b9a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4145170 0x7f93b40ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55972/0 (socket says 172.21.15.150:55972) 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93b9a00640 1 -- 172.21.15.150:0/416839367 learned_addr learned my addr 172.21.15.150:0/416839367 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.645+0000 7f93b9000640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b40b59f0 0x7f93b4141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b3e00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b40ce070 0x7f93b4141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b9a00640 1 -- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b40b59f0 msgr2=0x7f93b4141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b9a00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b40b59f0 0x7f93b4141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b9a00640 1 -- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b40ce070 msgr2=0x7f93b4141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b9a00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b40ce070 0x7f93b4141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b9a00640 1 -- 172.21.15.150:0/416839367 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b41f3b80 con 0x7f93b4145170 2024-04-15T15:25:17.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b3e00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b40ce070 0x7f93b4141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:17.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b9a00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4145170 0x7f93b40ac8b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c410 tx=0x7f93ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:17.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b0c00640 1 -- 172.21.15.150:0/416839367 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac011790 con 0x7f93b4145170 2024-04-15T15:25:17.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b0c00640 1 -- 172.21.15.150:0/416839367 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac017070 con 0x7f93b4145170 2024-04-15T15:25:17.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93b0c00640 1 -- 172.21.15.150:0/416839367 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac011bb0 con 0x7f93b4145170 2024-04-15T15:25:17.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.646+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41f3e60 con 0x7f93b4145170 2024-04-15T15:25:17.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.647+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b41f4420 con 0x7f93b4145170 2024-04-15T15:25:17.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.648+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b4068c80 con 0x7f93b4145170 2024-04-15T15:25:17.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.649+0000 7f93b0c00640 1 -- 172.21.15.150:0/416839367 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93ac01c460 con 0x7f93b4145170 2024-04-15T15:25:17.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.650+0000 7f93b0c00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:17.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.650+0000 7f93b0c00640 1 -- 172.21.15.150:0/416839367 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f93ac08b7e0 con 0x7f93b4145170 2024-04-15T15:25:17.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.650+0000 7f93b9000640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:17.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.651+0000 7f93b9000640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f939006c8f0 0x7f939006edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f93a40099c0 tx=0x7f93a4003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:17.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.654+0000 7f93b0c00640 1 -- 172.21.15.150:0/416839367 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93ac021020 con 0x7f93b4145170 2024-04-15T15:25:17.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.853+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mds_cache_quiesce_splitauth}] v 0) -- 0x7f93b40b3770 con 0x7f93b4145170 2024-04-15T15:25:17.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.858+0000 7f93b0c00640 1 -- 172.21.15.150:0/416839367 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_quiesce_splitauth}]=0 v12)=0 v12) ==== 128+0+0 (secure 0 0 0) 0x7f93ac059c10 con 0x7f93b4145170 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.861+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f939006c8f0 msgr2=0x7f939006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.861+0000 7f93bba00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f939006c8f0 0x7f939006edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f93a40099c0 tx=0x7f93a4003990 comp rx=0 tx=0).stop 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.861+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4145170 msgr2=0x7f93b40ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.861+0000 7f93bba00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4145170 0x7f93b40ac8b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c410 tx=0x7f93ac00c8d0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.862+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 shutdown_connections 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.862+0000 7f93bba00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b4145170 0x7f93b40ac8b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.862+0000 7f93bba00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b40ce070 0x7f93b4141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.862+0000 7f93bba00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.862+0000 7f93bba00640 1 --2- 172.21.15.150:0/416839367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93b40b59f0 0x7f93b4141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.862+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 >> 172.21.15.150:0/416839367 conn(0x7f93b40b2460 msgr2=0x7f93b40aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.862+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 shutdown_connections 2024-04-15T15:25:17.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:17.862+0000 7f93bba00640 1 -- 172.21.15.150:0/416839367 wait complete. 2024-04-15T15:25:17.884 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol_quiesce --mode=777 2024-04-15T15:25:18.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.097+0000 7f3ed1400640 1 Processor -- start 2024-04-15T15:25:18.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.097+0000 7f3ed1400640 1 -- start start 2024-04-15T15:25:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.099+0000 7f3ed1400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ecc141960 0x7f3ecc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.099+0000 7f3ed1400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3ecc15bbf0 0x7f3ecc15c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.099+0000 7f3ed1400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ecc15c980 con 0x7f3ecc0cff70 2024-04-15T15:25:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.099+0000 7f3ed1400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f3ecc0b46d0 con 0x7f3ecc141960 2024-04-15T15:25:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.099+0000 7f3ed1400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3ecc0b49b0 con 0x7f3ecc15bbf0 2024-04-15T15:25:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.099+0000 7f3eca800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ecc141960 0x7f3ecc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.099+0000 7f3eca800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ecc141960 0x7f3ecc0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55984/0 (socket says 172.21.15.150:55984) 2024-04-15T15:25:18.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.099+0000 7f3eca800640 1 -- 172.21.15.150:0/4080387823 learned_addr learned my addr 172.21.15.150:0/4080387823 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:18.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.100+0000 7f3ecb200640 1 --2- 172.21.15.150:0/4080387823 >> v2:172.21.15.187:3300/0 conn(0x7f3ecc15bbf0 0x7f3ecc15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.100+0000 7f3eca800640 1 -- 172.21.15.150:0/4080387823 >> v2:172.21.15.187:3300/0 conn(0x7f3ecc15bbf0 msgr2=0x7f3ecc15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:18.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.100+0000 7f3eca800640 1 --2- 172.21.15.150:0/4080387823 >> v2:172.21.15.187:3300/0 conn(0x7f3ecc15bbf0 0x7f3ecc15c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.100+0000 7f3eca800640 1 -- 172.21.15.150:0/4080387823 >> v1:172.21.15.187:6789/0 conn(0x7f3ecc0cff70 legacy=0x7f3ecc0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:18.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.100+0000 7f3eca800640 1 -- 172.21.15.150:0/4080387823 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc0b4c90 con 0x7f3ecc141960 2024-04-15T15:25:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.101+0000 7f3eca800640 1 --2- 172.21.15.150:0/4080387823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc141960 0x7f3ecc0b51b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0d0b70 tx=0x7f3ebc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e49679391401ae server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.102+0000 7f3ec9e00640 1 -- 172.21.15.150:0/4080387823 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc035020 con 0x7f3ecc141960 2024-04-15T15:25:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.102+0000 7f3ec9e00640 1 -- 172.21.15.150:0/4080387823 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc031870 con 0x7f3ecc141960 2024-04-15T15:25:18.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.102+0000 7f3ec9e00640 1 -- 172.21.15.150:0/4080387823 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc031c40 con 0x7f3ecc141960 2024-04-15T15:25:18.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.102+0000 7f3ed1400640 1 -- 172.21.15.150:0/4080387823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc141960 msgr2=0x7f3ecc0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.102+0000 7f3ed1400640 1 --2- 172.21.15.150:0/4080387823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc141960 0x7f3ecc0b51b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0d0b70 tx=0x7f3ebc02b910 comp rx=0 tx=0).stop 2024-04-15T15:25:18.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.103+0000 7f3ed1400640 1 -- 172.21.15.150:0/4080387823 shutdown_connections 2024-04-15T15:25:18.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.103+0000 7f3ed1400640 1 --2- 172.21.15.150:0/4080387823 >> v2:172.21.15.187:3300/0 conn(0x7f3ecc15bbf0 0x7f3ecc15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.103+0000 7f3ed1400640 1 --2- 172.21.15.150:0/4080387823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc141960 0x7f3ecc0b51b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.103+0000 7f3ed1400640 1 -- 172.21.15.150:0/4080387823 >> 172.21.15.150:0/4080387823 conn(0x7f3ecc0b2d20 msgr2=0x7f3ecc0b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:18.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.103+0000 7f3ed1400640 1 -- 172.21.15.150:0/4080387823 shutdown_connections 2024-04-15T15:25:18.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.103+0000 7f3ed1400640 1 -- 172.21.15.150:0/4080387823 wait complete. 2024-04-15T15:25:18.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.105+0000 7f3ed1400640 1 Processor -- start 2024-04-15T15:25:18.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.105+0000 7f3ed1400640 1 -- start start 2024-04-15T15:25:18.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3ed1400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc0cff70 0x7f3ecc1545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3ed1400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ecc141960 0x7f3ecc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3ed1400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ecc15bbf0 0x7f3ecc14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3ed1400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc15c720 con 0x7f3ecc0cff70 2024-04-15T15:25:18.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3ed1400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc15c5a0 con 0x7f3ecc141960 2024-04-15T15:25:18.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3ed1400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ecc06b930 con 0x7f3ecc15bbf0 2024-04-15T15:25:18.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3eca800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc0cff70 0x7f3ecc1545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3eca800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc0cff70 0x7f3ecc1545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55998/0 (socket says 172.21.15.150:55998) 2024-04-15T15:25:18.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3eca800640 1 -- 172.21.15.150:0/2887346270 learned_addr learned my addr 172.21.15.150:0/2887346270 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3eca800640 1 -- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ecc15bbf0 msgr2=0x7f3ecc14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3eca800640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ecc15bbf0 0x7f3ecc14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3eca800640 1 -- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ecc141960 msgr2=0x7f3ecc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3eca800640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ecc141960 0x7f3ecc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.106+0000 7f3eca800640 1 -- 172.21.15.150:0/2887346270 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc0b0d60 con 0x7f3ecc0cff70 2024-04-15T15:25:18.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.107+0000 7f3eca800640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc0cff70 0x7f3ecc1545a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc000c00 tx=0x7f3ebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.107+0000 7f3ec3400640 1 -- 172.21.15.150:0/2887346270 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc03ad60 con 0x7f3ecc0cff70 2024-04-15T15:25:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.107+0000 7f3ec3400640 1 -- 172.21.15.150:0/2887346270 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc0443f0 con 0x7f3ecc0cff70 2024-04-15T15:25:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.107+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc0b1040 con 0x7f3ecc0cff70 2024-04-15T15:25:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.107+0000 7f3ec3400640 1 -- 172.21.15.150:0/2887346270 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0447e0 con 0x7f3ecc0cff70 2024-04-15T15:25:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.107+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc0b1600 con 0x7f3ecc0cff70 2024-04-15T15:25:18.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.108+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ecc068db0 con 0x7f3ecc0cff70 2024-04-15T15:25:18.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.109+0000 7f3ec3400640 1 -- 172.21.15.150:0/2887346270 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ebc04b060 con 0x7f3ecc0cff70 2024-04-15T15:25:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.110+0000 7f3ec3400640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ea806c8f0 0x7f3ea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.110+0000 7f3ec3400640 1 -- 172.21.15.150:0/2887346270 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3ebc0addd0 con 0x7f3ecc0cff70 2024-04-15T15:25:18.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.111+0000 7f3ec1e00640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ea806c8f0 0x7f3ea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.112+0000 7f3ec1e00640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ea806c8f0 0x7f3ea806edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0b64c0 tx=0x7f3eb4007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.114+0000 7f3ec3400640 1 -- 172.21.15.150:0/2887346270 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ebc03f070 con 0x7f3ecc0cff70 2024-04-15T15:25:18.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.343+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f3ecc0b2510 con 0x7f3ea806c8f0 2024-04-15T15:25:18.358 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:25:18.357+0000 7fa5eea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:25:18.358 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:25:18.357+0000 7fa5eea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:25:18.358 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:25:18.357+0000 7fa5eea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:25:18.359 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:25:18.357+0000 7fa5eea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:25:18.359 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-15T15:25:18.357+0000 7fa5eea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-15T15:25:18.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.436+0000 7f3ec3400640 1 -- 172.21.15.150:0/2887346270 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ecc0b2510 con 0x7f3ea806c8f0 2024-04-15T15:25:18.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ea806c8f0 msgr2=0x7f3ea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ea806c8f0 0x7f3ea806edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0b64c0 tx=0x7f3eb4007990 comp rx=0 tx=0).stop 2024-04-15T15:25:18.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc0cff70 msgr2=0x7f3ecc1545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc0cff70 0x7f3ecc1545a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc000c00 tx=0x7f3ebc0046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 shutdown_connections 2024-04-15T15:25:18.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3ea806c8f0 0x7f3ea806edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ecc15bbf0 0x7f3ecc14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ecc141960 0x7f3ecc0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 --2- 172.21.15.150:0/2887346270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ecc0cff70 0x7f3ecc1545a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.439+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 >> 172.21.15.150:0/2887346270 conn(0x7f3ecc0b2d20 msgr2=0x7f3ecc06bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:18.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.440+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 shutdown_connections 2024-04-15T15:25:18.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.440+0000 7f3ed1400640 1 -- 172.21.15.150:0/2887346270 wait complete. 2024-04-15T15:25:18.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_quiesce 2024-04-15T15:25:18.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.671+0000 7f6f77000640 1 Processor -- start 2024-04-15T15:25:18.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.671+0000 7f6f77000640 1 -- start start 2024-04-15T15:25:18.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f77000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 0x7f6f700ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f77000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f700cd2d0 con 0x7f6f701461c0 2024-04-15T15:25:18.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f77000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f700cd5b0 con 0x7f6f700ce9e0 2024-04-15T15:25:18.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f77000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6f700cf1d0 con 0x7f6f700725f0 2024-04-15T15:25:18.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f75c00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6f701461c0 0x7f6f70071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50360/0 (socket says 172.21.15.150:50360) 2024-04-15T15:25:18.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f75c00640 1 -- 172.21.15.150:0/2065529663 learned_addr learned my addr 172.21.15.150:0/2065529663 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:18.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131329052 0 0) 0x7f6f700cd2d0 con 0x7f6f701461c0 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f58003520 con 0x7f6f701461c0 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.673+0000 7f6f75200640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 0x7f6f700ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1724951451 0 0) 0x7f6f58003520 con 0x7f6f701461c0 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 msgr2=0x7f6f700ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 0x7f6f700ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 >> v1:172.21.15.187:6789/0 conn(0x7f6f700ce9e0 legacy=0x7f6f7006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f700cd890 con 0x7f6f701461c0 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f6f64001670 con 0x7f6f701461c0 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 >> v1:172.21.15.150:6789/0 conn(0x7f6f701461c0 legacy=0x7f6f70071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f580039e0 0x7f6f58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 0x7f6f58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5800ba70 con 0x7f6f58004d40 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5800bbf0 con 0x7f6f580039e0 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5800bdf0 con 0x7f6f700725f0 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.674+0000 7f6f76600640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.675+0000 7f6f75c00640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f580039e0 0x7f6f58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.675+0000 7f6f75200640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 0x7f6f58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.675+0000 7f6f76600640 1 -- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 msgr2=0x7f6f58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.675+0000 7f6f76600640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 0x7f6f58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.675+0000 7f6f76600640 1 -- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f580039e0 msgr2=0x7f6f58003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.675+0000 7f6f76600640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f580039e0 0x7f6f58003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.675+0000 7f6f76600640 1 -- 172.21.15.150:0/2065529663 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f58003520 con 0x7f6f58004d40 2024-04-15T15:25:18.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.675+0000 7f6f75200640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700725f0 0x7f6f58004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:18.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.676+0000 7f6f76600640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f580051c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6f68001e20 tx=0x7f6f58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c6173f4b288bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.676+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68001a70 con 0x7f6f58004d40 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.676+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f68003b70 con 0x7f6f58004d40 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.676+0000 7f6f6fe00640 1 -- 172.21.15.150:0/2065529663 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f680050d0 con 0x7f6f58004d40 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.677+0000 7f6f77000640 1 -- 172.21.15.150:0/2065529663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 msgr2=0x7f6f580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.677+0000 7f6f77000640 1 --2- 172.21.15.150:0/2065529663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f580051c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6f68001e20 tx=0x7f6f58005830 comp rx=0 tx=0).stop 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.677+0000 7f6f76600640 1 -- 172.21.15.150:0/2065529663 reap_dead start 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.677+0000 7f6f77000640 1 -- 172.21.15.150:0/2065529663 shutdown_connections 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.677+0000 7f6f77000640 1 -- 172.21.15.150:0/2065529663 >> 172.21.15.150:0/2065529663 conn(0x7f6f7006b930 msgr2=0x7f6f7006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.677+0000 7f6f77000640 1 -- 172.21.15.150:0/2065529663 shutdown_connections 2024-04-15T15:25:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.677+0000 7f6f77000640 1 -- 172.21.15.150:0/2065529663 wait complete. 2024-04-15T15:25:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.679+0000 7f6f77000640 1 Processor -- start 2024-04-15T15:25:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.679+0000 7f6f77000640 1 -- start start 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f77000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f700ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f77000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700ae8b0 0x7f6f701f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f77000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f701f4af0 0x7f6f701f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f77000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6f700cee70 con 0x7f6f58004d40 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f77000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f700cd080 con 0x7f6f701f4af0 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f77000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6f700ce4a0 con 0x7f6f700ae8b0 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f75c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f700ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f75c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f700ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56014/0 (socket says 172.21.15.150:56014) 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f75c00640 1 -- 172.21.15.150:0/4253910577 learned_addr learned my addr 172.21.15.150:0/4253910577 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f75c00640 1 -- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700ae8b0 msgr2=0x7f6f701f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f75c00640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700ae8b0 0x7f6f701f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f75c00640 1 -- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f701f4af0 msgr2=0x7f6f701f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f75c00640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f701f4af0 0x7f6f701f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.680+0000 7f6f75c00640 1 -- 172.21.15.150:0/4253910577 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f700aedf0 con 0x7f6f58004d40 2024-04-15T15:25:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.681+0000 7f6f75c00640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f700ae370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400e820 tx=0x7f6f6400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.681+0000 7f6f6d600640 1 -- 172.21.15.150:0/4253910577 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400c7f0 con 0x7f6f58004d40 2024-04-15T15:25:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.681+0000 7f6f6d600640 1 -- 172.21.15.150:0/4253910577 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f64018070 con 0x7f6f58004d40 2024-04-15T15:25:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.681+0000 7f6f6d600640 1 -- 172.21.15.150:0/4253910577 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400cbd0 con 0x7f6f58004d40 2024-04-15T15:25:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.681+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f700af0d0 con 0x7f6f58004d40 2024-04-15T15:25:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.681+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f70159810 con 0x7f6f58004d40 2024-04-15T15:25:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.682+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f700755c0 con 0x7f6f58004d40 2024-04-15T15:25:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.683+0000 7f6f6d600640 1 -- 172.21.15.150:0/4253910577 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f6401d460 con 0x7f6f58004d40 2024-04-15T15:25:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.684+0000 7f6f6d600640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6f5006c8f0 0x7f6f5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.684+0000 7f6f6d600640 1 -- 172.21.15.150:0/4253910577 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6f6408bb20 con 0x7f6f58004d40 2024-04-15T15:25:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.685+0000 7f6f75200640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6f5006c8f0 0x7f6f5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:18.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.686+0000 7f6f75200640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6f5006c8f0 0x7f6f5006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0099e0 tx=0x7f6f5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:18.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.689+0000 7f6f6d600640 1 -- 172.21.15.150:0/4253910577 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f64059eb0 con 0x7f6f58004d40 2024-04-15T15:25:18.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.920+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "target": ["mon-mgr", ""]}) -- 0x7f6f70072e40 con 0x7f6f5006c8f0 2024-04-15T15:25:18.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.932+0000 7f6f6d600640 1 -- 172.21.15.150:0/4253910577 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+70 (secure 0 0 0) 0x7f6f70072e40 con 0x7f6f5006c8f0 2024-04-15T15:25:18.934 INFO:teuthology.orchestra.run.smithi150.stdout:/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:18.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.934+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6f5006c8f0 msgr2=0x7f6f5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.934+0000 7f6f77000640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6f5006c8f0 0x7f6f5006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0099e0 tx=0x7f6f5c008040 comp rx=0 tx=0).stop 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.934+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 msgr2=0x7f6f700ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.934+0000 7f6f77000640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f700ae370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400e820 tx=0x7f6f6400ece0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.935+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 shutdown_connections 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.935+0000 7f6f77000640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f701f4af0 0x7f6f701f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.935+0000 7f6f77000640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6f5006c8f0 0x7f6f5006edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.935+0000 7f6f77000640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f700ae8b0 0x7f6f701f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.935+0000 7f6f77000640 1 --2- 172.21.15.150:0/4253910577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f58004d40 0x7f6f700ae370 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.935+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 >> 172.21.15.150:0/4253910577 conn(0x7f6f7006b930 msgr2=0x7f6f700b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.935+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 shutdown_connections 2024-04-15T15:25:18.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:18.935+0000 7f6f77000640 1 -- 172.21.15.150:0/4253910577 wait complete. 2024-04-15T15:25:18.957 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:25:18.957 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:19.005 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:25:19.005 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:19.098 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:19.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.310+0000 7f693be00640 1 Processor -- start 2024-04-15T15:25:19.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.310+0000 7f693be00640 1 -- start start 2024-04-15T15:25:19.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.312+0000 7f693be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f693c072660 0x7f693c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.312+0000 7f693be00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c162940 con 0x7f693c148130 2024-04-15T15:25:19.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.312+0000 7f693be00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c162c20 con 0x7f693c162380 2024-04-15T15:25:19.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.312+0000 7f693be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f693c148510 con 0x7f693c072660 2024-04-15T15:25:19.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.312+0000 7f693aa00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f693c148130 0x7f693c071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50376/0 (socket says 172.21.15.150:50376) 2024-04-15T15:25:19.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.312+0000 7f693aa00640 1 -- 172.21.15.150:0/2589989555 learned_addr learned my addr 172.21.15.150:0/2589989555 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f693a000640 1 --2- 172.21.15.150:0/2589989555 >> v2:172.21.15.187:3300/0 conn(0x7f693c072660 0x7f693c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3448663412 0 0) 0x7f693c162940 con 0x7f693c148130 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6920003520 con 0x7f693c148130 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1163703794 0 0) 0x7f6920003520 con 0x7f693c148130 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 >> v2:172.21.15.187:3300/0 conn(0x7f693c072660 msgr2=0x7f693c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 --2- 172.21.15.150:0/2589989555 >> v2:172.21.15.187:3300/0 conn(0x7f693c072660 0x7f693c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 >> v1:172.21.15.187:6789/0 conn(0x7f693c162380 legacy=0x7f693c0af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c162f00 con 0x7f693c148130 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f692c001670 con 0x7f693c148130 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 >> v1:172.21.15.150:6789/0 conn(0x7f693c148130 legacy=0x7f693c071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200039b0 0x7f6920003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6920004800 0x7f6920004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200051c0 0x7f692000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f693c148510 con 0x7f69200051c0 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f692000c160 con 0x7f6920004800 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.313+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f692000c360 con 0x7f69200039b0 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.314+0000 7f693b400640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200051c0 0x7f692000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.314+0000 7f693aa00640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200039b0 0x7f6920003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.314+0000 7f693b400640 1 -- 172.21.15.150:0/2589989555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200039b0 msgr2=0x7f6920003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.314+0000 7f693b400640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200039b0 0x7f6920003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.314+0000 7f693a000640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6920004800 0x7f6920004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.314+0000 7f693b400640 1 -- 172.21.15.150:0/2589989555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6920004800 msgr2=0x7f6920004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.314+0000 7f693b400640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6920004800 0x7f6920004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.314+0000 7f693b400640 1 -- 172.21.15.150:0/2589989555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6920003520 con 0x7f69200051c0 2024-04-15T15:25:19.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.315+0000 7f693b400640 1 -- 172.21.15.150:0/2589989555 reap_dead start 2024-04-15T15:25:19.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.315+0000 7f693b400640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200051c0 0x7f692000b9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6934001e50 tx=0x7f6920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d8998b91317d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934001a70 con 0x7f69200051c0 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6934003b90 con 0x7f69200051c0 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f6939600640 1 -- 172.21.15.150:0/2589989555 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340050d0 con 0x7f69200051c0 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f693be00640 1 -- 172.21.15.150:0/2589989555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200051c0 msgr2=0x7f692000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f693be00640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200051c0 0x7f692000b9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6934001e50 tx=0x7f6920005830 comp rx=0 tx=0).stop 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f693be00640 1 -- 172.21.15.150:0/2589989555 shutdown_connections 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f693be00640 1 --2- 172.21.15.150:0/2589989555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200051c0 0x7f692000b9c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f693be00640 1 -- 172.21.15.150:0/2589989555 >> 172.21.15.150:0/2589989555 conn(0x7f693c06b930 msgr2=0x7f693c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f693be00640 1 -- 172.21.15.150:0/2589989555 shutdown_connections 2024-04-15T15:25:19.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.316+0000 7f693be00640 1 -- 172.21.15.150:0/2589989555 wait complete. 2024-04-15T15:25:19.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.318+0000 7f693be00640 1 Processor -- start 2024-04-15T15:25:19.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.318+0000 7f693be00640 1 -- start start 2024-04-15T15:25:19.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200051c0 0x7f693c13e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c0cd040 0x7f693c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f693c0cd4c0 0x7f693c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0aed10 con 0x7f693c0cd4c0 2024-04-15T15:25:19.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0aeb90 con 0x7f693c0cd040 2024-04-15T15:25:19.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f693c15ff50 con 0x7f69200051c0 2024-04-15T15:25:19.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f693c0cd4c0 0x7f693c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f693c0cd4c0 0x7f693c15e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56044/0 (socket says 172.21.15.150:56044) 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.319+0000 7f693b400640 1 -- 172.21.15.150:0/1404269201 learned_addr learned my addr 172.21.15.150:0/1404269201 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f693aa00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200051c0 0x7f693c13e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f693a000640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c0cd040 0x7f693c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f693b400640 1 -- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200051c0 msgr2=0x7f693c13e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f693b400640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200051c0 0x7f693c13e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f693b400640 1 -- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c0cd040 msgr2=0x7f693c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f693b400640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c0cd040 0x7f693c15dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f693b400640 1 -- 172.21.15.150:0/1404269201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c0cda20 con 0x7f693c0cd4c0 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f693b400640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f693c0cd4c0 0x7f693c15e170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6934001f70 tx=0x7f69340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f6932a00640 1 -- 172.21.15.150:0/1404269201 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934006d50 con 0x7f693c0cd4c0 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.320+0000 7f6932a00640 1 -- 172.21.15.150:0/1404269201 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6934007030 con 0x7f693c0cd4c0 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.321+0000 7f6932a00640 1 -- 172.21.15.150:0/1404269201 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934007420 con 0x7f693c0cd4c0 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.321+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c0cdd00 con 0x7f693c0cd4c0 2024-04-15T15:25:19.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.321+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c0ce2c0 con 0x7f693c0cd4c0 2024-04-15T15:25:19.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.322+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c0755c0 con 0x7f693c0cd4c0 2024-04-15T15:25:19.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.323+0000 7f6932a00640 1 -- 172.21.15.150:0/1404269201 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69340075c0 con 0x7f693c0cd4c0 2024-04-15T15:25:19.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.324+0000 7f6932a00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f691c06c920 0x7f691c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.324+0000 7f6932a00640 1 -- 172.21.15.150:0/1404269201 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f69340ad750 con 0x7f693c0cd4c0 2024-04-15T15:25:19.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.324+0000 7f693aa00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f691c06c920 0x7f691c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.326+0000 7f693aa00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f691c06c920 0x7f691c06ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f692c001200 tx=0x7f692c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:19.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.328+0000 7f6932a00640 1 -- 172.21.15.150:0/1404269201 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6934033030 con 0x7f693c0cd4c0 2024-04-15T15:25:19.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.421+0000 7f6932a00640 1 -- 172.21.15.150:0/1404269201 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f69340078b0 con 0x7f693c0cd4c0 2024-04-15T15:25:19.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.539+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f693c06f6d0 con 0x7f693c0cd4c0 2024-04-15T15:25:19.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.542+0000 7f6932a00640 1 -- 172.21.15.150:0/1404269201 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7f693407bc10 con 0x7f693c0cd4c0 2024-04-15T15:25:19.544 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:19.544 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":35,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:07.917174+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:19.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f691c06c920 msgr2=0x7f691c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f691c06c920 0x7f691c06ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f692c001200 tx=0x7f692c009040 comp rx=0 tx=0).stop 2024-04-15T15:25:19.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f693c0cd4c0 msgr2=0x7f693c15e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f693c0cd4c0 0x7f693c15e170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6934001f70 tx=0x7f69340014a0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 shutdown_connections 2024-04-15T15:25:19.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f693c0cd4c0 0x7f693c15e170 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c0cd040 0x7f693c15dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f691c06c920 0x7f691c06ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.544+0000 7f693be00640 1 --2- 172.21.15.150:0/1404269201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69200051c0 0x7f693c13e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.545+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 >> 172.21.15.150:0/1404269201 conn(0x7f693c06b930 msgr2=0x7f693c145720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:19.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.545+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 shutdown_connections 2024-04-15T15:25:19.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.545+0000 7f693be00640 1 -- 172.21.15.150:0/1404269201 wait complete. 2024-04-15T15:25:19.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:25:04.190900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:25:05.324314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:25:19.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:25:19.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.783+0000 7f6d77800640 1 Processor -- start 2024-04-15T15:25:19.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.784+0000 7f6d77800640 1 -- start start 2024-04-15T15:25:19.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.785+0000 7f6d77800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6d7015a440 0x7f6d7015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.785+0000 7f6d77800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7006fc80 0x7f6d700cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.785+0000 7f6d77800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d7015d520 con 0x7f6d700cd620 2024-04-15T15:25:19.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.785+0000 7f6d77800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f6d7006e280 con 0x7f6d7015a440 2024-04-15T15:25:19.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.785+0000 7f6d77800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d7015d800 con 0x7f6d7006fc80 2024-04-15T15:25:19.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.785+0000 7f6d74e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6d7015a440 0x7f6d7015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.786+0000 7f6d74e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6d7015a440 0x7f6d7015cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56046/0 (socket says 172.21.15.150:56046) 2024-04-15T15:25:19.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.786+0000 7f6d74e00640 1 -- 172.21.15.150:0/1534853644 learned_addr learned my addr 172.21.15.150:0/1534853644 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:19.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.786+0000 7f6d67e00640 1 --2- 172.21.15.150:0/1534853644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7006fc80 0x7f6d700cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.786+0000 7f6d74e00640 1 -- 172.21.15.150:0/1534853644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7006fc80 msgr2=0x7f6d700cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:19.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.786+0000 7f6d74e00640 1 --2- 172.21.15.150:0/1534853644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7006fc80 0x7f6d700cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.786+0000 7f6d74e00640 1 -- 172.21.15.150:0/1534853644 >> v1:172.21.15.187:6789/0 conn(0x7f6d700cd620 legacy=0x7f6d70072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:19.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.786+0000 7f6d74e00640 1 -- 172.21.15.150:0/1534853644 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7015dae0 con 0x7f6d7015a440 2024-04-15T15:25:19.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.787+0000 7f6d74e00640 1 --2- 172.21.15.150:0/1534853644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7015a440 0x7f6d7015cef0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009870 tx=0x7f6d6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1176eb68e6d71aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:19.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.788+0000 7f6d6fe00640 1 -- 172.21.15.150:0/1534853644 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60035020 con 0x7f6d7015a440 2024-04-15T15:25:19.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.788+0000 7f6d6fe00640 1 -- 172.21.15.150:0/1534853644 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d60031640 con 0x7f6d7015a440 2024-04-15T15:25:19.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.788+0000 7f6d6fe00640 1 -- 172.21.15.150:0/1534853644 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6002d040 con 0x7f6d7015a440 2024-04-15T15:25:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.788+0000 7f6d77800640 1 -- 172.21.15.150:0/1534853644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7015a440 msgr2=0x7f6d7015cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.788+0000 7f6d77800640 1 --2- 172.21.15.150:0/1534853644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7015a440 0x7f6d7015cef0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009870 tx=0x7f6d6002b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.789+0000 7f6d77800640 1 -- 172.21.15.150:0/1534853644 shutdown_connections 2024-04-15T15:25:19.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.789+0000 7f6d77800640 1 --2- 172.21.15.150:0/1534853644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7006fc80 0x7f6d700cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.789+0000 7f6d77800640 1 --2- 172.21.15.150:0/1534853644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7015a440 0x7f6d7015cef0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.789+0000 7f6d77800640 1 -- 172.21.15.150:0/1534853644 >> 172.21.15.150:0/1534853644 conn(0x7f6d700b2460 msgr2=0x7f6d700b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:19.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.789+0000 7f6d77800640 1 -- 172.21.15.150:0/1534853644 shutdown_connections 2024-04-15T15:25:19.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.789+0000 7f6d77800640 1 -- 172.21.15.150:0/1534853644 wait complete. 2024-04-15T15:25:19.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.790+0000 7f6d77800640 1 Processor -- start 2024-04-15T15:25:19.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.791+0000 7f6d77800640 1 -- start start 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.791+0000 7f6d77800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7006fc80 0x7f6d7013d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.791+0000 7f6d77800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d700cd620 0x7f6d70140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.791+0000 7f6d77800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7015a440 0x7f6d700b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.791+0000 7f6d77800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7006f510 con 0x7f6d7006fc80 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.791+0000 7f6d77800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7015ddc0 con 0x7f6d700cd620 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.791+0000 7f6d77800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d70072860 con 0x7f6d7015a440 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d74e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7006fc80 0x7f6d7013d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d74e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7006fc80 0x7f6d7013d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56060/0 (socket says 172.21.15.150:56060) 2024-04-15T15:25:19.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d74e00640 1 -- 172.21.15.150:0/1424230260 learned_addr learned my addr 172.21.15.150:0/1424230260 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:19.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d67e00640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d700cd620 0x7f6d70140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d75800640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7015a440 0x7f6d700b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d74e00640 1 -- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7015a440 msgr2=0x7f6d700b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d74e00640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7015a440 0x7f6d700b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d74e00640 1 -- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d700cd620 msgr2=0x7f6d70140440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:19.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d74e00640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d700cd620 0x7f6d70140440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:19.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.792+0000 7f6d74e00640 1 -- 172.21.15.150:0/1424230260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7013d7a0 con 0x7f6d7006fc80 2024-04-15T15:25:19.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.793+0000 7f6d74e00640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7006fc80 0x7f6d7013d260 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6d6002b930 tx=0x7f6d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:19.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.793+0000 7f6d6d600640 1 -- 172.21.15.150:0/1424230260 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60040070 con 0x7f6d7006fc80 2024-04-15T15:25:19.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.793+0000 7f6d6d600640 1 -- 172.21.15.150:0/1424230260 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d60039c10 con 0x7f6d7006fc80 2024-04-15T15:25:19.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.793+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7013e9b0 con 0x7f6d7006fc80 2024-04-15T15:25:19.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.793+0000 7f6d6d600640 1 -- 172.21.15.150:0/1424230260 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60035040 con 0x7f6d7006fc80 2024-04-15T15:25:19.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.793+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7013f250 con 0x7f6d7006fc80 2024-04-15T15:25:19.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.794+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d70068c80 con 0x7f6d7006fc80 2024-04-15T15:25:19.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.795+0000 7f6d6d600640 1 -- 172.21.15.150:0/1424230260 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6d60038020 con 0x7f6d7006fc80 2024-04-15T15:25:19.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.796+0000 7f6d6d600640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6d4c06c980 0x7f6d4c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:19.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.796+0000 7f6d6d600640 1 -- 172.21.15.150:0/1424230260 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6d600ad700 con 0x7f6d7006fc80 2024-04-15T15:25:19.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.797+0000 7f6d67e00640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6d4c06c980 0x7f6d4c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:19.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.798+0000 7f6d67e00640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6d4c06c980 0x7f6d4c06ee40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d70159e60 tx=0x7f6d58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:19.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.801+0000 7f6d6d600640 1 -- 172.21.15.150:0/1424230260 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d7013d7a0 con 0x7f6d7006fc80 2024-04-15T15:25:20.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:19.999+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d7013ed40 con 0x7f6d7006fc80 2024-04-15T15:25:20.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.001+0000 7f6d6d600640 1 -- 172.21.15.150:0/1424230260 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f6d6007bb40 con 0x7f6d7006fc80 2024-04-15T15:25:20.003 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:25:20.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6d4c06c980 msgr2=0x7f6d4c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6d4c06c980 0x7f6d4c06ee40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d70159e60 tx=0x7f6d58008040 comp rx=0 tx=0).stop 2024-04-15T15:25:20.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7006fc80 msgr2=0x7f6d7013d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7006fc80 0x7f6d7013d260 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6d6002b930 tx=0x7f6d600046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 shutdown_connections 2024-04-15T15:25:20.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6d4c06c980 0x7f6d4c06ee40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d7015a440 0x7f6d700b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d700cd620 0x7f6d70140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 --2- 172.21.15.150:0/1424230260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d7006fc80 0x7f6d7013d260 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.003+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 >> 172.21.15.150:0/1424230260 conn(0x7f6d700b2460 msgr2=0x7f6d700b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:20.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.004+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 shutdown_connections 2024-04-15T15:25:20.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.004+0000 7f6d77800640 1 -- 172.21.15.150:0/1424230260 wait complete. 2024-04-15T15:25:20.024 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:25:20.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.282+0000 7f9e37e00640 1 Processor -- start 2024-04-15T15:25:20.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.282+0000 7f9e37e00640 1 -- start start 2024-04-15T15:25:20.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.283+0000 7f9e37e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9e380b55f0 0x7f9e3815f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.283+0000 7f9e37e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e380b5a80 con 0x7f9e3815adc0 2024-04-15T15:25:20.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.283+0000 7f9e37e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e380cd2d0 con 0x7f9e38072770 2024-04-15T15:25:20.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.283+0000 7f9e37e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f9e380cd080 con 0x7f9e380b55f0 2024-04-15T15:25:20.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.284+0000 7f9e2e000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9e380b55f0 0x7f9e3815f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.284+0000 7f9e36a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9e3815adc0 0x7f9e380cec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50384/0 (socket says 172.21.15.150:50384) 2024-04-15T15:25:20.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.284+0000 7f9e36a00640 1 -- 172.21.15.150:0/81231196 learned_addr learned my addr 172.21.15.150:0/81231196 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:20.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.284+0000 7f9e2e000640 1 -- 172.21.15.150:0/81231196 >> v1:172.21.15.187:6789/0 conn(0x7f9e38072770 legacy=0x7f9e38072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:20.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.284+0000 7f9e2e000640 1 -- 172.21.15.150:0/81231196 >> v1:172.21.15.150:6789/0 conn(0x7f9e3815adc0 legacy=0x7f9e380cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.284+0000 7f9e2e000640 1 -- 172.21.15.150:0/81231196 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e380cd5b0 con 0x7f9e380b55f0 2024-04-15T15:25:20.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.284+0000 7f9e36000640 1 -- 172.21.15.150:0/81231196 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111001641 0 0) 0x7f9e380b5a80 con 0x7f9e3815adc0 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.285+0000 7f9e2e000640 1 --2- 172.21.15.150:0/81231196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e380b55f0 0x7f9e3815f2b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000a940 tx=0x7f9e2002cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db9e09ab5aa9df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e36000640 1 -- 172.21.15.150:0/81231196 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2003a070 con 0x7f9e380b55f0 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e36000640 1 -- 172.21.15.150:0/81231196 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2002db40 con 0x7f9e380b55f0 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e36000640 1 -- 172.21.15.150:0/81231196 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000b040 con 0x7f9e380b55f0 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e37e00640 1 -- 172.21.15.150:0/81231196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e380b55f0 msgr2=0x7f9e3815f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e37e00640 1 --2- 172.21.15.150:0/81231196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e380b55f0 0x7f9e3815f2b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000a940 tx=0x7f9e2002cd90 comp rx=0 tx=0).stop 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e37e00640 1 -- 172.21.15.150:0/81231196 shutdown_connections 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e37e00640 1 --2- 172.21.15.150:0/81231196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e380b55f0 0x7f9e3815f2b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e37e00640 1 -- 172.21.15.150:0/81231196 >> 172.21.15.150:0/81231196 conn(0x7f9e3806b930 msgr2=0x7f9e3806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:20.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.286+0000 7f9e37e00640 1 -- 172.21.15.150:0/81231196 shutdown_connections 2024-04-15T15:25:20.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.287+0000 7f9e37e00640 1 -- 172.21.15.150:0/81231196 wait complete. 2024-04-15T15:25:20.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.288+0000 7f9e37e00640 1 Processor -- start 2024-04-15T15:25:20.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.288+0000 7f9e37e00640 1 -- start start 2024-04-15T15:25:20.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.289+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e38072770 0x7f9e38151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.289+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e380b55f0 0x7f9e380ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.289+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e3815adc0 0x7f9e380aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.289+0000 7f9e37e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9e380cf360 con 0x7f9e3815adc0 2024-04-15T15:25:20.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.289+0000 7f9e37e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3806df20 con 0x7f9e380b55f0 2024-04-15T15:25:20.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.289+0000 7f9e37e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e380cf4e0 con 0x7f9e38072770 2024-04-15T15:25:20.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e37400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e3815adc0 0x7f9e380aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e37400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e3815adc0 0x7f9e380aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56074/0 (socket says 172.21.15.150:56074) 2024-04-15T15:25:20.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e37400640 1 -- 172.21.15.150:0/435936415 learned_addr learned my addr 172.21.15.150:0/435936415 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:20.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e36a00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e38072770 0x7f9e38151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e2e000640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e380b55f0 0x7f9e380ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e37400640 1 -- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e38072770 msgr2=0x7f9e38151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e37400640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e38072770 0x7f9e38151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e37400640 1 -- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e380b55f0 msgr2=0x7f9e380ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e37400640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e380b55f0 0x7f9e380ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.290+0000 7f9e37400640 1 -- 172.21.15.150:0/435936415 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e380cc800 con 0x7f9e3815adc0 2024-04-15T15:25:20.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.291+0000 7f9e37400640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e3815adc0 0x7f9e380aed90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c410 tx=0x7f9e3000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:20.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.291+0000 7f9e2fe00640 1 -- 172.21.15.150:0/435936415 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30011790 con 0x7f9e3815adc0 2024-04-15T15:25:20.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.291+0000 7f9e2fe00640 1 -- 172.21.15.150:0/435936415 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e30017070 con 0x7f9e3815adc0 2024-04-15T15:25:20.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.291+0000 7f9e2fe00640 1 -- 172.21.15.150:0/435936415 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30011bb0 con 0x7f9e3815adc0 2024-04-15T15:25:20.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.291+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e380ccae0 con 0x7f9e3815adc0 2024-04-15T15:25:20.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.291+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e380ac980 con 0x7f9e3815adc0 2024-04-15T15:25:20.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.292+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e3806f640 con 0x7f9e3815adc0 2024-04-15T15:25:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.293+0000 7f9e2fe00640 1 -- 172.21.15.150:0/435936415 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9e3001c460 con 0x7f9e3815adc0 2024-04-15T15:25:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.294+0000 7f9e2fe00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9e1406c980 0x7f9e1406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.294+0000 7f9e2fe00640 1 -- 172.21.15.150:0/435936415 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9e3008b770 con 0x7f9e3815adc0 2024-04-15T15:25:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.295+0000 7f9e36a00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9e1406c980 0x7f9e1406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.296+0000 7f9e36a00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9e1406c980 0x7f9e1406ee40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e28007ed0 tx=0x7f9e28007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:20.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.299+0000 7f9e2fe00640 1 -- 172.21.15.150:0/435936415 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e30021020 con 0x7f9e3815adc0 2024-04-15T15:25:20.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.496+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9e380cee60 con 0x7f9e3815adc0 2024-04-15T15:25:20.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.497+0000 7f9e2fe00640 1 -- 172.21.15.150:0/435936415 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f9e30059bb0 con 0x7f9e3815adc0 2024-04-15T15:25:20.499 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:25:20.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9e1406c980 msgr2=0x7f9e1406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9e1406c980 0x7f9e1406ee40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e28007ed0 tx=0x7f9e28007e60 comp rx=0 tx=0).stop 2024-04-15T15:25:20.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e3815adc0 msgr2=0x7f9e380aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e3815adc0 0x7f9e380aed90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c410 tx=0x7f9e3000c8d0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 shutdown_connections 2024-04-15T15:25:20.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e3815adc0 0x7f9e380aed90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e380b55f0 0x7f9e380ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9e1406c980 0x7f9e1406ee40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 --2- 172.21.15.150:0/435936415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e38072770 0x7f9e38151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.499+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 >> 172.21.15.150:0/435936415 conn(0x7f9e3806b930 msgr2=0x7f9e380b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:20.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.500+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 shutdown_connections 2024-04-15T15:25:20.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.500+0000 7f9e37e00640 1 -- 172.21.15.150:0/435936415 wait complete. 2024-04-15T15:25:20.521 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:25:20.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.735+0000 7f9728c00640 1 Processor -- start 2024-04-15T15:25:20.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.735+0000 7f9728c00640 1 -- start start 2024-04-15T15:25:20.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9728c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972415a440 0x7f972415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9728c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f972406fc80 0x7f97240cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9728c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f97240cd620 0x7f9724072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9728c00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f972406e280 con 0x7f97240cd620 2024-04-15T15:25:20.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9728c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f972415d520 con 0x7f972406fc80 2024-04-15T15:25:20.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9728c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f972415d800 con 0x7f972415a440 2024-04-15T15:25:20.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9722800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f97240cd620 0x7f9724072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9722800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f97240cd620 0x7f9724072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56078/0 (socket says 172.21.15.150:56078) 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9722800640 1 -- 172.21.15.150:0/4104836492 learned_addr learned my addr 172.21.15.150:0/4104836492 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f971be00640 1 --2- 172.21.15.150:0/4104836492 >> v2:172.21.15.187:3300/0 conn(0x7f972406fc80 0x7f97240cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.737+0000 7f9721e00640 1 --2- 172.21.15.150:0/4104836492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972415a440 0x7f972415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.738+0000 7f9722800640 1 -- 172.21.15.150:0/4104836492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972415a440 msgr2=0x7f972415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.738+0000 7f9722800640 1 --2- 172.21.15.150:0/4104836492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972415a440 0x7f972415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.738+0000 7f9722800640 1 -- 172.21.15.150:0/4104836492 >> v2:172.21.15.187:3300/0 conn(0x7f972406fc80 msgr2=0x7f97240cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.738+0000 7f9722800640 1 --2- 172.21.15.150:0/4104836492 >> v2:172.21.15.187:3300/0 conn(0x7f972406fc80 0x7f97240cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.738+0000 7f9722800640 1 -- 172.21.15.150:0/4104836492 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972415dae0 con 0x7f97240cd620 2024-04-15T15:25:20.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.739+0000 7f9722800640 1 --2- 172.21.15.150:0/4104836492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724072150 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f972406c9d0 tx=0x7f971c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c60b241ef2c0e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:20.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.739+0000 7f9721400640 1 -- 172.21.15.150:0/4104836492 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c02ca70 con 0x7f97240cd620 2024-04-15T15:25:20.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.739+0000 7f9721400640 1 -- 172.21.15.150:0/4104836492 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971c02cc10 con 0x7f97240cd620 2024-04-15T15:25:20.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.739+0000 7f9721400640 1 -- 172.21.15.150:0/4104836492 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c0343f0 con 0x7f97240cd620 2024-04-15T15:25:20.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 -- 172.21.15.150:0/4104836492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 msgr2=0x7f9724072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 --2- 172.21.15.150:0/4104836492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724072150 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f972406c9d0 tx=0x7f971c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 -- 172.21.15.150:0/4104836492 shutdown_connections 2024-04-15T15:25:20.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 --2- 172.21.15.150:0/4104836492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724072150 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 --2- 172.21.15.150:0/4104836492 >> v2:172.21.15.187:3300/0 conn(0x7f972406fc80 0x7f97240cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 --2- 172.21.15.150:0/4104836492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972415a440 0x7f972415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 -- 172.21.15.150:0/4104836492 >> 172.21.15.150:0/4104836492 conn(0x7f97240b2460 msgr2=0x7f97240b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:20.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 -- 172.21.15.150:0/4104836492 shutdown_connections 2024-04-15T15:25:20.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.740+0000 7f9728c00640 1 -- 172.21.15.150:0/4104836492 wait complete. 2024-04-15T15:25:20.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.742+0000 7f9728c00640 1 Processor -- start 2024-04-15T15:25:20.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.742+0000 7f9728c00640 1 -- start start 2024-04-15T15:25:20.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f9728c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972406fc80 0x7f97240abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f9728c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f9728c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f972415a440 0x7f9724143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f9728c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f972406f510 con 0x7f97240cd620 2024-04-15T15:25:20.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f9728c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f972415ddc0 con 0x7f972415a440 2024-04-15T15:25:20.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f9728c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9724072690 con 0x7f972406fc80 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f971be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f971be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56092/0 (socket says 172.21.15.150:56092) 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.743+0000 7f971be00640 1 -- 172.21.15.150:0/1779778541 learned_addr learned my addr 172.21.15.150:0/1779778541 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f9721e00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972406fc80 0x7f97240abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f9722800640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f972415a440 0x7f9724143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f971be00640 1 -- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972406fc80 msgr2=0x7f97240abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f971be00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972406fc80 0x7f97240abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f971be00640 1 -- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f972415a440 msgr2=0x7f9724143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f971be00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f972415a440 0x7f9724143d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f971be00640 1 -- 172.21.15.150:0/1779778541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97241f3650 con 0x7f97240cd620 2024-04-15T15:25:20.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f971be00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724143850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f970c00d800 tx=0x7f970c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:20.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.744+0000 7f971a000640 1 -- 172.21.15.150:0/1779778541 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c018020 con 0x7f97240cd620 2024-04-15T15:25:20.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.745+0000 7f971a000640 1 -- 172.21.15.150:0/1779778541 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970c0096a0 con 0x7f97240cd620 2024-04-15T15:25:20.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.745+0000 7f971a000640 1 -- 172.21.15.150:0/1779778541 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c009a70 con 0x7f97240cd620 2024-04-15T15:25:20.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.745+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97241f3930 con 0x7f97240cd620 2024-04-15T15:25:20.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.745+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97241f4410 con 0x7f97240cd620 2024-04-15T15:25:20.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.746+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9724068c80 con 0x7f97240cd620 2024-04-15T15:25:20.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.747+0000 7f971a000640 1 -- 172.21.15.150:0/1779778541 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f970c01b020 con 0x7f97240cd620 2024-04-15T15:25:20.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.748+0000 7f971a000640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f970006c980 0x7f970006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:20.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.748+0000 7f971a000640 1 -- 172.21.15.150:0/1779778541 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f970c08c110 con 0x7f97240cd620 2024-04-15T15:25:20.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.748+0000 7f9721e00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f970006c980 0x7f970006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:20.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.750+0000 7f9721e00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f970006c980 0x7f970006ee40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9714006fd0 tx=0x7f9714008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:20.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.753+0000 7f971a000640 1 -- 172.21.15.150:0/1779778541 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f970c022020 con 0x7f97240cd620 2024-04-15T15:25:20.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.956+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f97240afd90 con 0x7f97240cd620 2024-04-15T15:25:20.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.958+0000 7f971a000640 1 -- 172.21.15.150:0/1779778541 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f970c059f00 con 0x7f97240cd620 2024-04-15T15:25:20.960 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:25:20.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f970006c980 msgr2=0x7f970006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f970006c980 0x7f970006ee40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9714006fd0 tx=0x7f9714008040 comp rx=0 tx=0).stop 2024-04-15T15:25:20.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 msgr2=0x7f9724143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:20.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724143850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f970c00d800 tx=0x7f970c00dcc0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 shutdown_connections 2024-04-15T15:25:20.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f972415a440 0x7f9724143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97240cd620 0x7f9724143850 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f970006c980 0x7f970006ee40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 --2- 172.21.15.150:0/1779778541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f972406fc80 0x7f97240abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:20.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.960+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 >> 172.21.15.150:0/1779778541 conn(0x7f97240b2460 msgr2=0x7f97241449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:20.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.961+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 shutdown_connections 2024-04-15T15:25:20.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:20.961+0000 7f9728c00640 1 -- 172.21.15.150:0/1779778541 wait complete. 2024-04-15T15:25:20.982 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:25:21.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.193+0000 7f3b25400640 1 Processor -- start 2024-04-15T15:25:21.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.194+0000 7f3b25400640 1 -- start start 2024-04-15T15:25:21.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.195+0000 7f3b25400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3b20148110 0x7f3b2015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.195+0000 7f3b25400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2015ff20 con 0x7f3b2015f4e0 2024-04-15T15:25:21.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.195+0000 7f3b25400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2015f960 con 0x7f3b200b43b0 2024-04-15T15:25:21.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.195+0000 7f3b25400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3b2015fc40 con 0x7f3b20148110 2024-04-15T15:25:21.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.196+0000 7f3b1de00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3b2015f4e0 0x7f3b200aecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50392/0 (socket says 172.21.15.150:50392) 2024-04-15T15:25:21.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.196+0000 7f3b1de00640 1 -- 172.21.15.150:0/3531153820 learned_addr learned my addr 172.21.15.150:0/3531153820 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:21.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.196+0000 7f3b1e800640 1 --2- 172.21.15.150:0/3531153820 >> v2:172.21.15.187:3300/0 conn(0x7f3b20148110 0x7f3b2015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.196+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2184854332 0 0) 0x7f3b2015ff20 con 0x7f3b2015f4e0 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.196+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b04003520 con 0x7f3b2015f4e0 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1606909959 0 0) 0x7f3b04003520 con 0x7f3b2015f4e0 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 >> v2:172.21.15.187:3300/0 conn(0x7f3b20148110 msgr2=0x7f3b2015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 --2- 172.21.15.150:0/3531153820 >> v2:172.21.15.187:3300/0 conn(0x7f3b20148110 0x7f3b2015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 >> v1:172.21.15.187:6789/0 conn(0x7f3b200b43b0 legacy=0x7f3b20072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b200b4b20 con 0x7f3b2015f4e0 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f3b08001670 con 0x7f3b2015f4e0 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 >> v1:172.21.15.150:6789/0 conn(0x7f3b2015f4e0 legacy=0x7f3b200aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b040039b0 0x7f3b04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b040051c0 0x7f3b0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2015fc40 con 0x7f3b040051c0 2024-04-15T15:25:21.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0400c060 con 0x7f3b04004800 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0400c260 con 0x7f3b040039b0 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1f200640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b040051c0 0x7f3b0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.197+0000 7f3b1e800640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b040039b0 0x7f3b04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.198+0000 7f3b1de00640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.198+0000 7f3b1f200640 1 -- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b040039b0 msgr2=0x7f3b04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.198+0000 7f3b1f200640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b040039b0 0x7f3b04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.198+0000 7f3b1f200640 1 -- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b04004800 msgr2=0x7f3b04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.198+0000 7f3b1f200640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.198+0000 7f3b1f200640 1 -- 172.21.15.150:0/3531153820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b04003520 con 0x7f3b040051c0 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.198+0000 7f3b1de00640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:21.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.198+0000 7f3b1e800640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b040039b0 0x7f3b04003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.199+0000 7f3b1f200640 1 -- 172.21.15.150:0/3531153820 reap_dead start 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.199+0000 7f3b1f200640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b040051c0 0x7f3b0400b9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001e20 tx=0x7f3b04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a9a88e8c4f5e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.199+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18001a70 con 0x7f3b040051c0 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.199+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b18003b70 con 0x7f3b040051c0 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.199+0000 7f3b1d400640 1 -- 172.21.15.150:0/3531153820 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b180050d0 con 0x7f3b040051c0 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.200+0000 7f3b25400640 1 -- 172.21.15.150:0/3531153820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b040051c0 msgr2=0x7f3b0400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.200+0000 7f3b25400640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b040051c0 0x7f3b0400b9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001e20 tx=0x7f3b04005830 comp rx=0 tx=0).stop 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.200+0000 7f3b25400640 1 -- 172.21.15.150:0/3531153820 shutdown_connections 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.200+0000 7f3b25400640 1 --2- 172.21.15.150:0/3531153820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b040051c0 0x7f3b0400b9c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.200+0000 7f3b25400640 1 -- 172.21.15.150:0/3531153820 >> 172.21.15.150:0/3531153820 conn(0x7f3b200b2460 msgr2=0x7f3b200b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.200+0000 7f3b25400640 1 -- 172.21.15.150:0/3531153820 shutdown_connections 2024-04-15T15:25:21.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.200+0000 7f3b25400640 1 -- 172.21.15.150:0/3531153820 wait complete. 2024-04-15T15:25:21.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.202+0000 7f3b25400640 1 Processor -- start 2024-04-15T15:25:21.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.202+0000 7f3b25400640 1 -- start start 2024-04-15T15:25:21.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b25400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b040051c0 0x7f3b200cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b25400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20152cb0 0x7f3b2013d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b25400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b201e7280 0x7f3b201e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b25400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20148670 con 0x7f3b20152cb0 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b25400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b201484f0 con 0x7f3b040051c0 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b25400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2015efa0 con 0x7f3b201e7280 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b1de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20152cb0 0x7f3b2013d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b1de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20152cb0 0x7f3b2013d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56110/0 (socket says 172.21.15.150:56110) 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b1de00640 1 -- 172.21.15.150:0/2942292519 learned_addr learned my addr 172.21.15.150:0/2942292519 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b1de00640 1 -- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b201e7280 msgr2=0x7f3b201e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b1de00640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b201e7280 0x7f3b201e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b1de00640 1 -- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b040051c0 msgr2=0x7f3b200cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b1de00640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b040051c0 0x7f3b200cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.203+0000 7f3b1de00640 1 -- 172.21.15.150:0/2942292519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b200cd890 con 0x7f3b20152cb0 2024-04-15T15:25:21.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.204+0000 7f3b1de00640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20152cb0 0x7f3b2013d380 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800d7e0 tx=0x7f3b0800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:21.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.204+0000 7f3b16a00640 1 -- 172.21.15.150:0/2942292519 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08019070 con 0x7f3b20152cb0 2024-04-15T15:25:21.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.204+0000 7f3b16a00640 1 -- 172.21.15.150:0/2942292519 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b080074e0 con 0x7f3b20152cb0 2024-04-15T15:25:21.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.204+0000 7f3b16a00640 1 -- 172.21.15.150:0/2942292519 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08004a00 con 0x7f3b20152cb0 2024-04-15T15:25:21.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.204+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b200cdb70 con 0x7f3b20152cb0 2024-04-15T15:25:21.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.204+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b201e4cd0 con 0x7f3b20152cb0 2024-04-15T15:25:21.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.205+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b20068c80 con 0x7f3b20152cb0 2024-04-15T15:25:21.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.206+0000 7f3b16a00640 1 -- 172.21.15.150:0/2942292519 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3b0801c070 con 0x7f3b20152cb0 2024-04-15T15:25:21.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.207+0000 7f3b16a00640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3afc06c9b0 0x7f3afc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.207+0000 7f3b16a00640 1 -- 172.21.15.150:0/2942292519 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3b0808b7d0 con 0x7f3b20152cb0 2024-04-15T15:25:21.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.208+0000 7f3b1e800640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3afc06c9b0 0x7f3afc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.209+0000 7f3b1e800640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3afc06c9b0 0x7f3afc06ee70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3b100099e0 tx=0x7f3b10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:21.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.212+0000 7f3b16a00640 1 -- 172.21.15.150:0/2942292519 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b08059a70 con 0x7f3b20152cb0 2024-04-15T15:25:21.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.426+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b201436c0 con 0x7f3b20152cb0 2024-04-15T15:25:21.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.427+0000 7f3b16a00640 1 -- 172.21.15.150:0/2942292519 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f3b08059370 con 0x7f3b20152cb0 2024-04-15T15:25:21.429 INFO:teuthology.orchestra.run.smithi150.stderr:listed 0 entries 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3afc06c9b0 msgr2=0x7f3afc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3afc06c9b0 0x7f3afc06ee70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3b100099e0 tx=0x7f3b10008040 comp rx=0 tx=0).stop 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20152cb0 msgr2=0x7f3b2013d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20152cb0 0x7f3b2013d380 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800d7e0 tx=0x7f3b0800dca0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 shutdown_connections 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f3afc06c9b0 0x7f3afc06ee70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b201e7280 0x7f3b201e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20152cb0 0x7f3b2013d380 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 --2- 172.21.15.150:0/2942292519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b040051c0 0x7f3b200cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 >> 172.21.15.150:0/2942292519 conn(0x7f3b200b2460 msgr2=0x7f3b200b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:21.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 shutdown_connections 2024-04-15T15:25:21.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.431+0000 7f3b25400640 1 -- 172.21.15.150:0/2942292519 wait complete. 2024-04-15T15:25:21.452 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-15T15:25:21.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:21.452 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:21.557 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:25:21.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:21.558 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:25:21.608 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:25:21.608 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:21.666 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:25:21.666 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:21.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:21.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.940+0000 7f1ec5800640 1 Processor -- start 2024-04-15T15:25:21.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.940+0000 7f1ec5800640 1 -- start start 2024-04-15T15:25:21.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.942+0000 7f1ec5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec00d3e20 0x7f1ec00d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.942+0000 7f1ec5800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ec015ee00 0x7f1ec00cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.942+0000 7f1ec5800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec00b89f0 con 0x7f1ec00cf6f0 2024-04-15T15:25:21.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.942+0000 7f1ec5800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec00cfba0 con 0x7f1ec015ee00 2024-04-15T15:25:21.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.942+0000 7f1ec5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00d2b10 con 0x7f1ec00d3e20 2024-04-15T15:25:21.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.943+0000 7f1ebea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec00d3e20 0x7f1ec00d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.943+0000 7f1ebe000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ec015ee00 0x7f1ec00cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.943+0000 7f1ebe000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ec015ee00 0x7f1ec00cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:58118/0 (socket says 172.21.15.150:58118) 2024-04-15T15:25:21.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.943+0000 7f1ebe000640 1 -- 172.21.15.150:0/1505580839 learned_addr learned my addr 172.21.15.150:0/1505580839 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:21.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.943+0000 7f1ebe000640 1 -- 172.21.15.150:0/1505580839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec00d3e20 msgr2=0x7f1ec00d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:21.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.943+0000 7f1ebe000640 1 --2- 172.21.15.150:0/1505580839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec00d3e20 0x7f1ec00d4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.943+0000 7f1ebe000640 1 -- 172.21.15.150:0/1505580839 >> v1:172.21.15.187:6789/0 conn(0x7f1ec00cf6f0 legacy=0x7f1ec00b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.943+0000 7f1ebe000640 1 -- 172.21.15.150:0/1505580839 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec00d4740 con 0x7f1ec015ee00 2024-04-15T15:25:21.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.945+0000 7f1ebe000640 1 --2- 172.21.15.150:0/1505580839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec015ee00 0x7f1ec00cf1b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009870 tx=0x7f1ea802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1069a296cce3d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:21.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.945+0000 7f1ebd600640 1 -- 172.21.15.150:0/1505580839 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea802f840 con 0x7f1ec015ee00 2024-04-15T15:25:21.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.945+0000 7f1ebd600640 1 -- 172.21.15.150:0/1505580839 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea802f9e0 con 0x7f1ec015ee00 2024-04-15T15:25:21.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.945+0000 7f1ebd600640 1 -- 172.21.15.150:0/1505580839 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea802fdb0 con 0x7f1ec015ee00 2024-04-15T15:25:21.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.946+0000 7f1ec5800640 1 -- 172.21.15.150:0/1505580839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec015ee00 msgr2=0x7f1ec00cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.946+0000 7f1ec5800640 1 --2- 172.21.15.150:0/1505580839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec015ee00 0x7f1ec00cf1b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009870 tx=0x7f1ea802ba20 comp rx=0 tx=0).stop 2024-04-15T15:25:21.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.946+0000 7f1ec5800640 1 -- 172.21.15.150:0/1505580839 shutdown_connections 2024-04-15T15:25:21.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.946+0000 7f1ec5800640 1 --2- 172.21.15.150:0/1505580839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec015ee00 0x7f1ec00cf1b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.946+0000 7f1ec5800640 1 --2- 172.21.15.150:0/1505580839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec00d3e20 0x7f1ec00d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.946+0000 7f1ec5800640 1 -- 172.21.15.150:0/1505580839 >> 172.21.15.150:0/1505580839 conn(0x7f1ec006d470 msgr2=0x7f1ec006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:21.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.946+0000 7f1ec5800640 1 -- 172.21.15.150:0/1505580839 shutdown_connections 2024-04-15T15:25:21.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.946+0000 7f1ec5800640 1 -- 172.21.15.150:0/1505580839 wait complete. 2024-04-15T15:25:21.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.948+0000 7f1ec5800640 1 Processor -- start 2024-04-15T15:25:21.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.948+0000 7f1ec5800640 1 -- start start 2024-04-15T15:25:21.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ec5800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec00cf6f0 0x7f1ec0159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ec5800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00d3e20 0x7f1ec0159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ec5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec015ee00 0x7f1ec00cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ec5800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00d2d60 con 0x7f1ec00d3e20 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ec5800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00d33b0 con 0x7f1ec00cf6f0 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ec5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00d3040 con 0x7f1ec015ee00 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ebe000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00d3e20 0x7f1ec0159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ebe000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00d3e20 0x7f1ec0159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56126/0 (socket says 172.21.15.150:56126) 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ebe000640 1 -- 172.21.15.150:0/879676982 learned_addr learned my addr 172.21.15.150:0/879676982 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.949+0000 7f1ebea00640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec00cf6f0 0x7f1ec0159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ebf400640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec015ee00 0x7f1ec00cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ebe000640 1 -- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec015ee00 msgr2=0x7f1ec00cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ebe000640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec015ee00 0x7f1ec00cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ebe000640 1 -- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec00cf6f0 msgr2=0x7f1ec0159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ebe000640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec00cf6f0 0x7f1ec0159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:21.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ebe000640 1 -- 172.21.15.150:0/879676982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec0159590 con 0x7f1ec00d3e20 2024-04-15T15:25:21.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ebf400640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec015ee00 0x7f1ec00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:21.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ebe000640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00d3e20 0x7f1ec0159800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009870 tx=0x7f1ea8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:21.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1eb6a00640 1 -- 172.21.15.150:0/879676982 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8037bc0 con 0x7f1ec00d3e20 2024-04-15T15:25:21.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1eb6a00640 1 -- 172.21.15.150:0/879676982 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea8046070 con 0x7f1ec00d3e20 2024-04-15T15:25:21.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1eb6a00640 1 -- 172.21.15.150:0/879676982 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8041400 con 0x7f1ec00d3e20 2024-04-15T15:25:21.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.950+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec015a170 con 0x7f1ec00d3e20 2024-04-15T15:25:21.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.951+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec015a730 con 0x7f1ec00d3e20 2024-04-15T15:25:21.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.952+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec0069400 con 0x7f1ec00d3e20 2024-04-15T15:25:21.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.953+0000 7f1eb6a00640 1 -- 172.21.15.150:0/879676982 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1ea8040070 con 0x7f1ec00d3e20 2024-04-15T15:25:21.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.953+0000 7f1eb6a00640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1e9c06c980 0x7f1e9c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:21.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.954+0000 7f1eb6a00640 1 -- 172.21.15.150:0/879676982 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1ea80ae4c0 con 0x7f1ec00d3e20 2024-04-15T15:25:21.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.954+0000 7f1ebea00640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1e9c06c980 0x7f1e9c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:21.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.955+0000 7f1ebea00640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1e9c06c980 0x7f1e9c06ee40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00cb7d0 tx=0x7f1eb000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:21.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:21.958+0000 7f1eb6a00640 1 -- 172.21.15.150:0/879676982 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ec0069400 con 0x7f1ec00d3e20 2024-04-15T15:25:22.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.160+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ec0068c80 con 0x7f1ec00d3e20 2024-04-15T15:25:22.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.163+0000 7f1eb6a00640 1 -- 172.21.15.150:0/879676982 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7f1ea8033030 con 0x7f1ec00d3e20 2024-04-15T15:25:22.165 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:22.165 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":35,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:07.917174+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.165+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1e9c06c980 msgr2=0x7f1e9c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.165+0000 7f1ec5800640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1e9c06c980 0x7f1e9c06ee40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00cb7d0 tx=0x7f1eb000b040 comp rx=0 tx=0).stop 2024-04-15T15:25:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.165+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00d3e20 msgr2=0x7f1ec0159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.165+0000 7f1ec5800640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00d3e20 0x7f1ec0159800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009870 tx=0x7f1ea8004730 comp rx=0 tx=0).stop 2024-04-15T15:25:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.166+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 shutdown_connections 2024-04-15T15:25:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.166+0000 7f1ec5800640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1e9c06c980 0x7f1e9c06ee40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.166+0000 7f1ec5800640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ec015ee00 0x7f1ec00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.166+0000 7f1ec5800640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00d3e20 0x7f1ec0159800 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.166+0000 7f1ec5800640 1 --2- 172.21.15.150:0/879676982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ec00cf6f0 0x7f1ec0159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.166+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 >> 172.21.15.150:0/879676982 conn(0x7f1ec006d470 msgr2=0x7f1ec00b7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.166+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 shutdown_connections 2024-04-15T15:25:22.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.166+0000 7f1ec5800640 1 -- 172.21.15.150:0/879676982 wait complete. 2024-04-15T15:25:22.187 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:25:04.190900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:25:05.324314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:25:22.187 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:25:22.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.403+0000 7ff5e5a00640 1 Processor -- start 2024-04-15T15:25:22.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.403+0000 7ff5e5a00640 1 -- start start 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.405+0000 7ff5e5a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 0x7ff5e006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.405+0000 7ff5e5a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff5e00b59b0 0x7ff5e00b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.405+0000 7ff5e5a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e00cea20 con 0x7ff5e015d8f0 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.405+0000 7ff5e5a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e00cecd0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.405+0000 7ff5e5a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e00cefb0 con 0x7ff5e015d510 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.405+0000 7ff5e5000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff5e00b59b0 0x7ff5e00b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.405+0000 7ff5e5000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff5e00b59b0 0x7ff5e00b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56130/0 (socket says 172.21.15.150:56130) 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.405+0000 7ff5e5000640 1 -- 172.21.15.150:0/1103358208 learned_addr learned my addr 172.21.15.150:0/1103358208 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.406+0000 7ff5dfe00640 1 --2- 172.21.15.150:0/1103358208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 0x7ff5e006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.406+0000 7ff5e5000640 1 -- 172.21.15.150:0/1103358208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 msgr2=0x7ff5e006ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:22.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.406+0000 7ff5e5000640 1 --2- 172.21.15.150:0/1103358208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 0x7ff5e006ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.406+0000 7ff5e5000640 1 -- 172.21.15.150:0/1103358208 >> v1:172.21.15.187:6789/0 conn(0x7ff5e015d8f0 legacy=0x7ff5e00ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.406+0000 7ff5e5000640 1 -- 172.21.15.150:0/1103358208 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e006dfa0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.407+0000 7ff5e5000640 1 --2- 172.21.15.150:0/1103358208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e00b5e40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009e50 tx=0x7ff5d802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d5f78f778a53d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:22.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.407+0000 7ff5dea00640 1 -- 172.21.15.150:0/1103358208 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d802cb00 con 0x7ff5e00b59b0 2024-04-15T15:25:22.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5dea00640 1 -- 172.21.15.150:0/1103358208 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d802cca0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5dea00640 1 -- 172.21.15.150:0/1103358208 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8034610 con 0x7ff5e00b59b0 2024-04-15T15:25:22.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5e5a00640 1 -- 172.21.15.150:0/1103358208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 msgr2=0x7ff5e00b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/1103358208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e00b5e40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009e50 tx=0x7ff5d802bf60 comp rx=0 tx=0).stop 2024-04-15T15:25:22.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5e5a00640 1 -- 172.21.15.150:0/1103358208 shutdown_connections 2024-04-15T15:25:22.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/1103358208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e00b5e40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/1103358208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 0x7ff5e006ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5e5a00640 1 -- 172.21.15.150:0/1103358208 >> 172.21.15.150:0/1103358208 conn(0x7ff5e00b23a0 msgr2=0x7ff5e00b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:22.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5e5a00640 1 -- 172.21.15.150:0/1103358208 shutdown_connections 2024-04-15T15:25:22.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.408+0000 7ff5e5a00640 1 -- 172.21.15.150:0/1103358208 wait complete. 2024-04-15T15:25:22.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.410+0000 7ff5e5a00640 1 Processor -- start 2024-04-15T15:25:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.410+0000 7ff5e5a00640 1 -- start start 2024-04-15T15:25:22.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5e5a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e01527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5e5a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 0x7ff5e01488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5e5a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5e00b51b0 0x7ff5e014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5e5a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e006efc0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5e5a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e006e560 con 0x7ff5e00b51b0 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5e5a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e0152450 con 0x7ff5e015d510 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5dfe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e01527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5dfe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e01527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56138/0 (socket says 172.21.15.150:56138) 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.411+0000 7ff5dfe00640 1 -- 172.21.15.150:0/4066174971 learned_addr learned my addr 172.21.15.150:0/4066174971 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5df400640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 0x7ff5e01488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5e5000640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5e00b51b0 0x7ff5e014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5dfe00640 1 -- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 msgr2=0x7ff5e01488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5dfe00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 0x7ff5e01488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5dfe00640 1 -- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5e00b51b0 msgr2=0x7ff5e014fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5dfe00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5e00b51b0 0x7ff5e014fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5dfe00640 1 -- 172.21.15.150:0/4066174971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e0152dc0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5dfe00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e01527e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000d830 tx=0x7ff5d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:22.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.412+0000 7ff5d7e00640 1 -- 172.21.15.150:0/4066174971 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000b6c0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.413+0000 7ff5d7e00640 1 -- 172.21.15.150:0/4066174971 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d0019070 con 0x7ff5e00b59b0 2024-04-15T15:25:22.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.413+0000 7ff5d7e00640 1 -- 172.21.15.150:0/4066174971 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000bac0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.413+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e01e6ea0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.413+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e01e7460 con 0x7ff5e00b59b0 2024-04-15T15:25:22.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.414+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e00755c0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.415+0000 7ff5d7e00640 1 -- 172.21.15.150:0/4066174971 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff5d001e460 con 0x7ff5e00b59b0 2024-04-15T15:25:22.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.416+0000 7ff5d7e00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff5c006c980 0x7ff5c006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.416+0000 7ff5d7e00640 1 -- 172.21.15.150:0/4066174971 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7ff5d008c290 con 0x7ff5e00b59b0 2024-04-15T15:25:22.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.417+0000 7ff5df400640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff5c006c980 0x7ff5c006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.418+0000 7ff5df400640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff5c006c980 0x7ff5c006ee40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc007fd0 tx=0x7ff5cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:22.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.421+0000 7ff5d7e00640 1 -- 172.21.15.150:0/4066174971 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5d005a760 con 0x7ff5e00b59b0 2024-04-15T15:25:22.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.622+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5e015cab0 con 0x7ff5e00b59b0 2024-04-15T15:25:22.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.622+0000 7ff5d7e00640 1 -- 172.21.15.150:0/4066174971 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7ff5d005a060 con 0x7ff5e00b59b0 2024-04-15T15:25:22.624 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:25:22.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff5c006c980 msgr2=0x7ff5c006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff5c006c980 0x7ff5c006ee40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc007fd0 tx=0x7ff5cc009040 comp rx=0 tx=0).stop 2024-04-15T15:25:22.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 msgr2=0x7ff5e01527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e01527e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000d830 tx=0x7ff5d000dcf0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 shutdown_connections 2024-04-15T15:25:22.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5e00b51b0 0x7ff5e014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff5c006c980 0x7ff5c006ee40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5e015d510 0x7ff5e01488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 --2- 172.21.15.150:0/4066174971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e00b59b0 0x7ff5e01527e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 >> 172.21.15.150:0/4066174971 conn(0x7ff5e00b23a0 msgr2=0x7ff5e015aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:22.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 shutdown_connections 2024-04-15T15:25:22.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.625+0000 7ff5e5a00640 1 -- 172.21.15.150:0/4066174971 wait complete. 2024-04-15T15:25:22.646 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:25:22.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.861+0000 7f65b3e00640 1 Processor -- start 2024-04-15T15:25:22.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.861+0000 7f65b3e00640 1 -- start start 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65b3e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f65b41478b0 0x7f65b40afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65b3e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b406f920 0x7f65b41626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65b3e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b40aebd0 con 0x7f65b4147c90 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65b3e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f65b40b0130 con 0x7f65b41478b0 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65b3e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65b40aee50 con 0x7f65b406f920 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65abe00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f65b4147c90 0x7f65b406f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50408/0 (socket says 172.21.15.150:50408) 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65abe00640 1 -- 172.21.15.150:0/3807287030 learned_addr learned my addr 172.21.15.150:0/3807287030 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65b3400640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b406f920 0x7f65b41626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65b2a00640 1 --2- 172.21.15.150:0/3807287030 >> v2:172.21.15.187:3300/0 conn(0x7f65b41478b0 0x7f65b40afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.863+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226723955 0 0) 0x7f659c001110 con 0x7f65b4147c90 2024-04-15T15:25:22.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6598003520 con 0x7f65b4147c90 2024-04-15T15:25:22.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 155389582 0 0) 0x7f6598003520 con 0x7f65b4147c90 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b406f920 msgr2=0x7f65b41626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b406f920 0x7f65b41626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 >> v2:172.21.15.187:3300/0 conn(0x7f65b41478b0 msgr2=0x7f65b40afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 --2- 172.21.15.150:0/3807287030 >> v2:172.21.15.187:3300/0 conn(0x7f65b41478b0 0x7f65b40afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b40af110 con 0x7f65b4147c90 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f659c001840 con 0x7f65b4147c90 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 >> v1:172.21.15.150:6789/0 conn(0x7f65b4147c90 legacy=0x7f65b406f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65980039b0 0x7f6598003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6598004800 0x7f6598004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b406f920 0x7f65980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40b0130 con 0x7f65980039b0 2024-04-15T15:25:22.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40aee50 con 0x7f6598004800 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f659800b9c0 con 0x7f65b406f920 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.864+0000 7f65b2a00640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65980039b0 0x7f6598003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.865+0000 7f65b2a00640 1 -- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b406f920 msgr2=0x7f65980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.865+0000 7f65b2a00640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b406f920 0x7f65980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.865+0000 7f65b2a00640 1 -- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6598004800 msgr2=0x7f6598004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.865+0000 7f65b2a00640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6598004800 0x7f6598004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.865+0000 7f65b2a00640 1 -- 172.21.15.150:0/3807287030 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6598003520 con 0x7f65980039b0 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.866+0000 7f65b2a00640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65980039b0 0x7f6598003dd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009ac0 tx=0x7f6598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677b98dd28f4f946 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.866+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4039070 con 0x7f65980039b0 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.866+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a402c710 con 0x7f65980039b0 2024-04-15T15:25:22.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.866+0000 7f65b2000640 1 -- 172.21.15.150:0/3807287030 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4009690 con 0x7f65980039b0 2024-04-15T15:25:22.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.867+0000 7f65b3e00640 1 -- 172.21.15.150:0/3807287030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65980039b0 msgr2=0x7f6598003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.867+0000 7f65b3e00640 1 --2- 172.21.15.150:0/3807287030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65980039b0 0x7f6598003dd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009ac0 tx=0x7f6598005830 comp rx=0 tx=0).stop 2024-04-15T15:25:22.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.867+0000 7f65b3400640 1 -- 172.21.15.150:0/3807287030 reap_dead start 2024-04-15T15:25:22.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.867+0000 7f65b3e00640 1 -- 172.21.15.150:0/3807287030 shutdown_connections 2024-04-15T15:25:22.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.867+0000 7f65b3e00640 1 -- 172.21.15.150:0/3807287030 >> 172.21.15.150:0/3807287030 conn(0x7f65b406b930 msgr2=0x7f65b406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:22.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.867+0000 7f65b3e00640 1 -- 172.21.15.150:0/3807287030 shutdown_connections 2024-04-15T15:25:22.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.867+0000 7f65b3e00640 1 -- 172.21.15.150:0/3807287030 wait complete. 2024-04-15T15:25:22.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.869+0000 7f65b3e00640 1 Processor -- start 2024-04-15T15:25:22.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.869+0000 7f65b3e00640 1 -- start start 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65b3e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65980039b0 0x7f65b413d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65b3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b41e87c0 0x7f65b40b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65b3e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b41e8c20 0x7f65b40b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65b3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65b41626c0 con 0x7f65b41e87c0 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65b3e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40af6d0 con 0x7f65980039b0 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65b3e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65b4162840 con 0x7f65b41e8c20 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65abe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b41e87c0 0x7f65b40b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65abe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b41e87c0 0x7f65b40b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56152/0 (socket says 172.21.15.150:56152) 2024-04-15T15:25:22.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.870+0000 7f65abe00640 1 -- 172.21.15.150:0/2082414580 learned_addr learned my addr 172.21.15.150:0/2082414580 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65b3400640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b41e8c20 0x7f65b40b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65b2a00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65980039b0 0x7f65b413d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65abe00640 1 -- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b41e8c20 msgr2=0x7f65b40b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65abe00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b41e8c20 0x7f65b40b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65abe00640 1 -- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65980039b0 msgr2=0x7f65b413d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65abe00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65980039b0 0x7f65b413d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65abe00640 1 -- 172.21.15.150:0/2082414580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b413e5a0 con 0x7f65b41e87c0 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65b3400640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b41e8c20 0x7f65b40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65b2a00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65980039b0 0x7f65b413d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:22.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65abe00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b41e87c0 0x7f65b40b1320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f659c00c8a0 tx=0x7f659c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:22.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65aaa00640 1 -- 172.21.15.150:0/2082414580 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c018070 con 0x7f65b41e87c0 2024-04-15T15:25:22.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65aaa00640 1 -- 172.21.15.150:0/2082414580 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c0074e0 con 0x7f65b41e87c0 2024-04-15T15:25:22.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65aaa00640 1 -- 172.21.15.150:0/2082414580 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c004a20 con 0x7f65b41e87c0 2024-04-15T15:25:22.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.871+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b413e880 con 0x7f65b41e87c0 2024-04-15T15:25:22.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.872+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b413ee40 con 0x7f65b41e87c0 2024-04-15T15:25:22.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.873+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b40afbf0 con 0x7f65b41e87c0 2024-04-15T15:25:22.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.874+0000 7f65aaa00640 1 -- 172.21.15.150:0/2082414580 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f659c01b070 con 0x7f65b41e87c0 2024-04-15T15:25:22.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.874+0000 7f65aaa00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f659006c980 0x7f659006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:22.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.875+0000 7f65aaa00640 1 -- 172.21.15.150:0/2082414580 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f659c08b1d0 con 0x7f65b41e87c0 2024-04-15T15:25:22.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.875+0000 7f65b2a00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f659006c980 0x7f659006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:22.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.876+0000 7f65b2a00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f659006c980 0x7f659006ee40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f65a4036000 tx=0x7f65a4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:22.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:22.879+0000 7f65aaa00640 1 -- 172.21.15.150:0/2082414580 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65b40afbf0 con 0x7f65b41e87c0 2024-04-15T15:25:23.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.087+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f65b40cc9d0 con 0x7f65b41e87c0 2024-04-15T15:25:23.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.088+0000 7f65aaa00640 1 -- 172.21.15.150:0/2082414580 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f65b40aebd0 con 0x7f65b41e87c0 2024-04-15T15:25:23.090 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:25:23.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f659006c980 msgr2=0x7f659006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f659006c980 0x7f659006ee40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f65a4036000 tx=0x7f65a4030000 comp rx=0 tx=0).stop 2024-04-15T15:25:23.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b41e87c0 msgr2=0x7f65b40b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b41e87c0 0x7f65b40b1320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f659c00c8a0 tx=0x7f659c00cd60 comp rx=0 tx=0).stop 2024-04-15T15:25:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 shutdown_connections 2024-04-15T15:25:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f659006c980 0x7f659006ee40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b41e8c20 0x7f65b40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b41e87c0 0x7f65b40b1320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 --2- 172.21.15.150:0/2082414580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65980039b0 0x7f65b413d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 >> 172.21.15.150:0/2082414580 conn(0x7f65b406b930 msgr2=0x7f65b4162c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 shutdown_connections 2024-04-15T15:25:23.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.090+0000 7f65b3e00640 1 -- 172.21.15.150:0/2082414580 wait complete. 2024-04-15T15:25:23.113 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:25:23.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.328+0000 7f5aba200640 1 Processor -- start 2024-04-15T15:25:23.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.328+0000 7f5aba200640 1 -- start start 2024-04-15T15:25:23.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5aba200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ab40b54b0 0x7f5ab40b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5aba200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab40b4410 con 0x7f5ab4145170 2024-04-15T15:25:23.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5aba200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab40b46f0 con 0x7f5ab406f260 2024-04-15T15:25:23.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5aba200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab4145550 con 0x7f5ab40b54b0 2024-04-15T15:25:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5ab3400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f5ab4145170 0x7f5ab40cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:50424/0 (socket says 172.21.15.150:50424) 2024-04-15T15:25:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5ab2a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ab40b54b0 0x7f5ab40b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5ab3400640 1 -- 172.21.15.150:0/3465303025 learned_addr learned my addr 172.21.15.150:0/3465303025 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5ab2a00640 1 -- 172.21.15.150:0/3465303025 >> v1:172.21.15.187:6789/0 conn(0x7f5ab406f260 legacy=0x7f5ab406f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.330+0000 7f5ab2a00640 1 -- 172.21.15.150:0/3465303025 >> v1:172.21.15.150:6789/0 conn(0x7f5ab4145170 legacy=0x7f5ab40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.331+0000 7f5ab2a00640 1 -- 172.21.15.150:0/3465303025 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40b49d0 con 0x7f5ab40b54b0 2024-04-15T15:25:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.331+0000 7f5ab2000640 1 -- 172.21.15.150:0/3465303025 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234797565 0 0) 0x7f5ab40b4410 con 0x7f5ab4145170 2024-04-15T15:25:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.332+0000 7f5ab2a00640 1 --2- 172.21.15.150:0/3465303025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab40b5940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5ab41456d0 tx=0x7f5a9c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7df2abbd9635824 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.332+0000 7f5ab2000640 1 -- 172.21.15.150:0/3465303025 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c039070 con 0x7f5ab40b54b0 2024-04-15T15:25:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.332+0000 7f5ab2000640 1 -- 172.21.15.150:0/3465303025 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c02cc60 con 0x7f5ab40b54b0 2024-04-15T15:25:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.332+0000 7f5ab2000640 1 -- 172.21.15.150:0/3465303025 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c034660 con 0x7f5ab40b54b0 2024-04-15T15:25:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.333+0000 7f5aba200640 1 -- 172.21.15.150:0/3465303025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 msgr2=0x7f5ab40b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.333+0000 7f5aba200640 1 --2- 172.21.15.150:0/3465303025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab40b5940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5ab41456d0 tx=0x7f5a9c02beb0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.333+0000 7f5aba200640 1 -- 172.21.15.150:0/3465303025 shutdown_connections 2024-04-15T15:25:23.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.333+0000 7f5aba200640 1 --2- 172.21.15.150:0/3465303025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab40b5940 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.333+0000 7f5aba200640 1 -- 172.21.15.150:0/3465303025 >> 172.21.15.150:0/3465303025 conn(0x7f5ab40b2460 msgr2=0x7f5ab40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:23.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.333+0000 7f5aba200640 1 -- 172.21.15.150:0/3465303025 shutdown_connections 2024-04-15T15:25:23.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.333+0000 7f5aba200640 1 -- 172.21.15.150:0/3465303025 wait complete. 2024-04-15T15:25:23.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.334+0000 7f5aba200640 1 Processor -- start 2024-04-15T15:25:23.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.335+0000 7f5aba200640 1 -- start start 2024-04-15T15:25:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.335+0000 7f5aba200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ab406f260 0x7f5ab40ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5aba200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab4130e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5aba200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ab4145170 0x7f5ab41413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5aba200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40ce240 con 0x7f5ab40b54b0 2024-04-15T15:25:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5aba200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40b4f90 con 0x7f5ab4145170 2024-04-15T15:25:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5aba200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab40b4030 con 0x7f5ab406f260 2024-04-15T15:25:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab2a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab4130e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab2a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab4130e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56160/0 (socket says 172.21.15.150:56160) 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab2a00640 1 -- 172.21.15.150:0/2951648525 learned_addr learned my addr 172.21.15.150:0/2951648525 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab3400640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ab406f260 0x7f5ab40ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab3e00640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ab4145170 0x7f5ab41413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab2a00640 1 -- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ab406f260 msgr2=0x7f5ab40ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab2a00640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ab406f260 0x7f5ab40ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab2a00640 1 -- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ab4145170 msgr2=0x7f5ab41413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab2a00640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ab4145170 0x7f5ab41413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab2a00640 1 -- 172.21.15.150:0/2951648525 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40acb30 con 0x7f5ab40b54b0 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.336+0000 7f5ab3e00640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ab4145170 0x7f5ab41413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.337+0000 7f5ab2a00640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab4130e10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c037040 tx=0x7f5a9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:23.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.337+0000 7f5aab400640 1 -- 172.21.15.150:0/2951648525 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c039040 con 0x7f5ab40b54b0 2024-04-15T15:25:23.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.337+0000 7f5aab400640 1 -- 172.21.15.150:0/2951648525 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c003b90 con 0x7f5ab40b54b0 2024-04-15T15:25:23.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.337+0000 7f5aab400640 1 -- 172.21.15.150:0/2951648525 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c004030 con 0x7f5ab40b54b0 2024-04-15T15:25:23.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.337+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab40aba90 con 0x7f5ab40b54b0 2024-04-15T15:25:23.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.337+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab40ac050 con 0x7f5ab40b54b0 2024-04-15T15:25:23.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.338+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab4068c80 con 0x7f5ab40b54b0 2024-04-15T15:25:23.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.339+0000 7f5aab400640 1 -- 172.21.15.150:0/2951648525 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5a9c02cb20 con 0x7f5ab40b54b0 2024-04-15T15:25:23.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.340+0000 7f5aab400640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5a9006c980 0x7f5a9006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.340+0000 7f5aab400640 1 -- 172.21.15.150:0/2951648525 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5a9c0aef40 con 0x7f5ab40b54b0 2024-04-15T15:25:23.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.341+0000 7f5ab3400640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5a9006c980 0x7f5a9006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.342+0000 7f5ab3400640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5a9006c980 0x7f5a9006ee40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4000f60 tx=0x7f5aa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:23.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.345+0000 7f5aab400640 1 -- 172.21.15.150:0/2951648525 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a9c07d1f0 con 0x7f5ab40b54b0 2024-04-15T15:25:23.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.545+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5ab415bb80 con 0x7f5ab40b54b0 2024-04-15T15:25:23.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.546+0000 7f5aab400640 1 -- 172.21.15.150:0/2951648525 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f5a9c07de20 con 0x7f5ab40b54b0 2024-04-15T15:25:23.548 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.548+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5a9006c980 msgr2=0x7f5a9006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5a9006c980 0x7f5a9006ee40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4000f60 tx=0x7f5aa4009040 comp rx=0 tx=0).stop 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 msgr2=0x7f5ab4130e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab4130e10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c037040 tx=0x7f5a9c004a40 comp rx=0 tx=0).stop 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 shutdown_connections 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ab4145170 0x7f5ab41413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab40b54b0 0x7f5ab4130e10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f5a9006c980 0x7f5a9006ee40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 --2- 172.21.15.150:0/2951648525 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ab406f260 0x7f5ab40ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 >> 172.21.15.150:0/2951648525 conn(0x7f5ab40b2460 msgr2=0x7f5ab40b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:23.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 shutdown_connections 2024-04-15T15:25:23.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.549+0000 7f5aba200640 1 -- 172.21.15.150:0/2951648525 wait complete. 2024-04-15T15:25:23.570 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:25:23.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.773+0000 7ff228800640 1 Processor -- start 2024-04-15T15:25:23.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.774+0000 7ff228800640 1 -- start start 2024-04-15T15:25:23.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.775+0000 7ff228800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff22015ff50 0x7ff2200d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.775+0000 7ff228800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff2200d0f60 0x7ff220070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.775+0000 7ff228800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff22009baa0 con 0x7ff22009b500 2024-04-15T15:25:23.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.775+0000 7ff228800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7ff22009c060 con 0x7ff2200d0f60 2024-04-15T15:25:23.777 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.775+0000 7ff228800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff22009bd80 con 0x7ff22015ff50 2024-04-15T15:25:23.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.776+0000 7ff226800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff2200d0f60 0x7ff220070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.776+0000 7ff226800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff2200d0f60 0x7ff220070cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56176/0 (socket says 172.21.15.150:56176) 2024-04-15T15:25:23.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.776+0000 7ff226800640 1 -- 172.21.15.150:0/3724604121 learned_addr learned my addr 172.21.15.150:0/3724604121 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:23.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.776+0000 7ff225e00640 1 --2- 172.21.15.150:0/3724604121 >> v2:172.21.15.187:3300/0 conn(0x7ff22015ff50 0x7ff2200d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.776+0000 7ff226800640 1 -- 172.21.15.150:0/3724604121 >> v2:172.21.15.187:3300/0 conn(0x7ff22015ff50 msgr2=0x7ff2200d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:23.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.776+0000 7ff226800640 1 --2- 172.21.15.150:0/3724604121 >> v2:172.21.15.187:3300/0 conn(0x7ff22015ff50 0x7ff2200d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.776+0000 7ff226800640 1 -- 172.21.15.150:0/3724604121 >> v1:172.21.15.187:6789/0 conn(0x7ff22009b500 legacy=0x7ff2200d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:23.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.776+0000 7ff226800640 1 -- 172.21.15.150:0/3724604121 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2200704d0 con 0x7ff2200d0f60 2024-04-15T15:25:23.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.777+0000 7ff226800640 1 --2- 172.21.15.150:0/3724604121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220070cd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff220073420 tx=0x7ff21c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5040ed04ec0f6b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:23.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff224a00640 1 -- 172.21.15.150:0/3724604121 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c02c9f0 con 0x7ff2200d0f60 2024-04-15T15:25:23.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff224a00640 1 -- 172.21.15.150:0/3724604121 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21c02cb90 con 0x7ff2200d0f60 2024-04-15T15:25:23.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff224a00640 1 -- 172.21.15.150:0/3724604121 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0343f0 con 0x7ff2200d0f60 2024-04-15T15:25:23.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff228800640 1 -- 172.21.15.150:0/3724604121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 msgr2=0x7ff220070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff228800640 1 --2- 172.21.15.150:0/3724604121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220070cd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff220073420 tx=0x7ff21c02be50 comp rx=0 tx=0).stop 2024-04-15T15:25:23.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff228800640 1 -- 172.21.15.150:0/3724604121 shutdown_connections 2024-04-15T15:25:23.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff228800640 1 --2- 172.21.15.150:0/3724604121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220070cd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff228800640 1 --2- 172.21.15.150:0/3724604121 >> v2:172.21.15.187:3300/0 conn(0x7ff22015ff50 0x7ff2200d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff228800640 1 -- 172.21.15.150:0/3724604121 >> 172.21.15.150:0/3724604121 conn(0x7ff2200d26c0 msgr2=0x7ff2200d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:23.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.778+0000 7ff228800640 1 -- 172.21.15.150:0/3724604121 shutdown_connections 2024-04-15T15:25:23.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.779+0000 7ff228800640 1 -- 172.21.15.150:0/3724604121 wait complete. 2024-04-15T15:25:23.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.780+0000 7ff228800640 1 Processor -- start 2024-04-15T15:25:23.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.780+0000 7ff228800640 1 -- start start 2024-04-15T15:25:23.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.781+0000 7ff228800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff22009b500 0x7ff2201f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.781+0000 7ff228800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.781+0000 7ff228800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff22015ff50 0x7ff2200c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.781+0000 7ff228800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200ce530 con 0x7ff2200d0f60 2024-04-15T15:25:23.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.781+0000 7ff228800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff220160330 con 0x7ff22015ff50 2024-04-15T15:25:23.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.781+0000 7ff228800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff220071210 con 0x7ff22009b500 2024-04-15T15:25:23.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.781+0000 7ff225400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56182/0 (socket says 172.21.15.150:56182) 2024-04-15T15:25:23.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225400640 1 -- 172.21.15.150:0/3356675205 learned_addr learned my addr 172.21.15.150:0/3356675205 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225e00640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff22009b500 0x7ff2201f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff226800640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff22015ff50 0x7ff2200c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225400640 1 -- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff22009b500 msgr2=0x7ff2201f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225400640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff22009b500 0x7ff2201f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225400640 1 -- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff22015ff50 msgr2=0x7ff2200c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225400640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff22015ff50 0x7ff2200c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225400640 1 -- 172.21.15.150:0/3356675205 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2201f4a60 con 0x7ff2200d0f60 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.782+0000 7ff225400640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220154af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00c460 tx=0x7ff20c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.783+0000 7ff21a000640 1 -- 172.21.15.150:0/3356675205 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c016020 con 0x7ff2200d0f60 2024-04-15T15:25:23.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.783+0000 7ff21a000640 1 -- 172.21.15.150:0/3356675205 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff20c012870 con 0x7ff2200d0f60 2024-04-15T15:25:23.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.783+0000 7ff21a000640 1 -- 172.21.15.150:0/3356675205 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c012c60 con 0x7ff2200d0f60 2024-04-15T15:25:23.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.783+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2201f4d40 con 0x7ff2200d0f60 2024-04-15T15:25:23.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.783+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2201f5300 con 0x7ff2200d0f60 2024-04-15T15:25:23.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.784+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff220068c80 con 0x7ff2200d0f60 2024-04-15T15:25:23.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.785+0000 7ff21a000640 1 -- 172.21.15.150:0/3356675205 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff20c019020 con 0x7ff2200d0f60 2024-04-15T15:25:23.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.786+0000 7ff21a000640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff20006c9b0 0x7ff20006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:23.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.786+0000 7ff21a000640 1 -- 172.21.15.150:0/3356675205 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7ff20c08baa0 con 0x7ff2200d0f60 2024-04-15T15:25:23.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.787+0000 7ff225e00640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff20006c9b0 0x7ff20006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:23.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.788+0000 7ff225e00640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff20006c9b0 0x7ff20006ee70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff214006fd0 tx=0x7ff214008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:23.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:23.790+0000 7ff21a000640 1 -- 172.21.15.150:0/3356675205 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff20c020020 con 0x7ff2200d0f60 2024-04-15T15:25:24.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.002+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff2200d15b0 con 0x7ff2200d0f60 2024-04-15T15:25:24.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.003+0000 7ff21a000640 1 -- 172.21.15.150:0/3356675205 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7ff20c0182c0 con 0x7ff2200d0f60 2024-04-15T15:25:24.004 INFO:teuthology.orchestra.run.smithi150.stderr:listed 0 entries 2024-04-15T15:25:24.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff20006c9b0 msgr2=0x7ff20006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff20006c9b0 0x7ff20006ee70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff214006fd0 tx=0x7ff214008040 comp rx=0 tx=0).stop 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 msgr2=0x7ff220154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220154af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00c460 tx=0x7ff20c00c920 comp rx=0 tx=0).stop 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 shutdown_connections 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff22015ff50 0x7ff2200c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2200d0f60 0x7ff220154af0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff20006c9b0 0x7ff20006ee70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 --2- 172.21.15.150:0/3356675205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff22009b500 0x7ff2201f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 >> 172.21.15.150:0/3356675205 conn(0x7ff2200d26c0 msgr2=0x7ff2200cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 shutdown_connections 2024-04-15T15:25:24.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.006+0000 7ff228800640 1 -- 172.21.15.150:0/3356675205 wait complete. 2024-04-15T15:25:24.027 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-15T15:25:24.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:24.027 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:24.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:25:24.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:24.141 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:25:24.196 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:25:24.196 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:24.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052773,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:25:24.251 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:25:24.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:24.251 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:25:24.299 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-15T15:25:24.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:25:24.301 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:25:24.301 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:24.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052761,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:25:24.357 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:25:24.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:24.357 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:25:24.408 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-15T15:25:24.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:25:24.410 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:25:24.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.621+0000 7f9d9be00640 1 Processor -- start 2024-04-15T15:25:24.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.621+0000 7f9d9be00640 1 -- start start 2024-04-15T15:25:24.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.622+0000 7f9d9be00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9d941461f0 0x7f9d940ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:24.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d9be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d940b3fe0 0x7f9d9415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:24.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d9be00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d940b4be0 con 0x7f9d940cedf0 2024-04-15T15:25:24.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d9be00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f9d940a9640 con 0x7f9d941461f0 2024-04-15T15:25:24.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d9be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9d940cf250 con 0x7f9d940b3fe0 2024-04-15T15:25:24.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d99400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9d941461f0 0x7f9d940ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:24.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d99400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9d941461f0 0x7f9d940ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56190/0 (socket says 172.21.15.150:56190) 2024-04-15T15:25:24.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d99400640 1 -- 172.21.15.150:0/951752492 learned_addr learned my addr 172.21.15.150:0/951752492 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:24.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d99e00640 1 --2- 172.21.15.150:0/951752492 >> v2:172.21.15.187:3300/0 conn(0x7f9d940b3fe0 0x7f9d9415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:24.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d99400640 1 -- 172.21.15.150:0/951752492 >> v2:172.21.15.187:3300/0 conn(0x7f9d940b3fe0 msgr2=0x7f9d9415d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:24.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d99400640 1 --2- 172.21.15.150:0/951752492 >> v2:172.21.15.187:3300/0 conn(0x7f9d940b3fe0 0x7f9d9415d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.623+0000 7f9d99400640 1 -- 172.21.15.150:0/951752492 >> v1:172.21.15.187:6789/0 conn(0x7f9d940cedf0 legacy=0x7f9d940b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:24.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.624+0000 7f9d99400640 1 -- 172.21.15.150:0/951752492 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d940b4620 con 0x7f9d941461f0 2024-04-15T15:25:24.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.625+0000 7f9d99400640 1 --2- 172.21.15.150:0/951752492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d940ce8b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9d940cf9a0 tx=0x7f9d8802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a72c8cf21f0f9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:24.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.625+0000 7f9d93e00640 1 -- 172.21.15.150:0/951752492 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88035020 con 0x7f9d941461f0 2024-04-15T15:25:24.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.625+0000 7f9d93e00640 1 -- 172.21.15.150:0/951752492 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d88031870 con 0x7f9d941461f0 2024-04-15T15:25:24.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.625+0000 7f9d93e00640 1 -- 172.21.15.150:0/951752492 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88031c40 con 0x7f9d941461f0 2024-04-15T15:25:24.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.626+0000 7f9d9be00640 1 -- 172.21.15.150:0/951752492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 msgr2=0x7f9d940ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:24.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.626+0000 7f9d9be00640 1 --2- 172.21.15.150:0/951752492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d940ce8b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9d940cf9a0 tx=0x7f9d8802b910 comp rx=0 tx=0).stop 2024-04-15T15:25:24.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.626+0000 7f9d9be00640 1 -- 172.21.15.150:0/951752492 shutdown_connections 2024-04-15T15:25:24.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.626+0000 7f9d9be00640 1 --2- 172.21.15.150:0/951752492 >> v2:172.21.15.187:3300/0 conn(0x7f9d940b3fe0 0x7f9d9415d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.626+0000 7f9d9be00640 1 --2- 172.21.15.150:0/951752492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d940ce8b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.626+0000 7f9d9be00640 1 -- 172.21.15.150:0/951752492 >> 172.21.15.150:0/951752492 conn(0x7f9d940b27b0 msgr2=0x7f9d940b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:24.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.626+0000 7f9d9be00640 1 -- 172.21.15.150:0/951752492 shutdown_connections 2024-04-15T15:25:24.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.626+0000 7f9d9be00640 1 -- 172.21.15.150:0/951752492 wait complete. 2024-04-15T15:25:24.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.628+0000 7f9d9be00640 1 Processor -- start 2024-04-15T15:25:24.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.628+0000 7f9d9be00640 1 -- start start 2024-04-15T15:25:24.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d9be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d940b3fe0 0x7f9d9413f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:24.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d9be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d940cedf0 0x7f9d9413a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:24.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d9be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d9406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:24.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d9be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9415d290 con 0x7f9d941461f0 2024-04-15T15:25:24.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d9be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9415cfb0 con 0x7f9d940b3fe0 2024-04-15T15:25:24.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d9be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d941465d0 con 0x7f9d940cedf0 2024-04-15T15:25:24.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d99e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d9406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:24.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d99e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d9406eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56206/0 (socket says 172.21.15.150:56206) 2024-04-15T15:25:24.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d99e00640 1 -- 172.21.15.150:0/2315735834 learned_addr learned my addr 172.21.15.150:0/2315735834 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:24.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d99400640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d940b3fe0 0x7f9d9413f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:24.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.629+0000 7f9d98a00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d940cedf0 0x7f9d9413a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:24.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d99e00640 1 -- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d940cedf0 msgr2=0x7f9d9413a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:24.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d99e00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d940cedf0 0x7f9d9413a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d99e00640 1 -- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d940b3fe0 msgr2=0x7f9d9413f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d99e00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d940b3fe0 0x7f9d9413f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d99e00640 1 -- 172.21.15.150:0/2315735834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9413fe10 con 0x7f9d941461f0 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d99400640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d940b3fe0 0x7f9d9413f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d99e00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d9406eeb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c007e40 tx=0x7f9d8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d91600640 1 -- 172.21.15.150:0/2315735834 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c014620 con 0x7f9d941461f0 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d91600640 1 -- 172.21.15.150:0/2315735834 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8c019070 con 0x7f9d941461f0 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d91600640 1 -- 172.21.15.150:0/2315735834 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c014a20 con 0x7f9d941461f0 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d941400f0 con 0x7f9d941461f0 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.630+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941406b0 con 0x7f9d941461f0 2024-04-15T15:25:24.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.631+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d94068c80 con 0x7f9d941461f0 2024-04-15T15:25:24.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.633+0000 7f9d91600640 1 -- 172.21.15.150:0/2315735834 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9d8c014bc0 con 0x7f9d941461f0 2024-04-15T15:25:24.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.633+0000 7f9d91600640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7406c980 0x7f9d7406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:24.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.634+0000 7f9d91600640 1 -- 172.21.15.150:0/2315735834 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9d8c08c550 con 0x7f9d941461f0 2024-04-15T15:25:24.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.634+0000 7f9d99400640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7406c980 0x7f9d7406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:24.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.635+0000 7f9d99400640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7406c980 0x7f9d7406ee40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d88036000 tx=0x7f9d88008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:24.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.638+0000 7f9d91600640 1 -- 172.21.15.150:0/2315735834 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d8c05a900 con 0x7f9d941461f0 2024-04-15T15:25:24.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.882+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9d9406d1e0 con 0x7f9d941461f0 2024-04-15T15:25:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.883+0000 7f9d91600640 1 -- 172.21.15.150:0/2315735834 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f9d8c05a200 con 0x7f9d941461f0 2024-04-15T15:25:24.885 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:25:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7406c980 msgr2=0x7f9d7406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7406c980 0x7f9d7406ee40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d88036000 tx=0x7f9d88008030 comp rx=0 tx=0).stop 2024-04-15T15:25:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 msgr2=0x7f9d9406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d9406eeb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c007e40 tx=0x7f9d8c00e4b0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 shutdown_connections 2024-04-15T15:25:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d941461f0 0x7f9d9406eeb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9d7406c980 0x7f9d7406ee40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d940cedf0 0x7f9d9413a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 --2- 172.21.15.150:0/2315735834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d940b3fe0 0x7f9d9413f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 >> 172.21.15.150:0/2315735834 conn(0x7f9d940b27b0 msgr2=0x7f9d940c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 shutdown_connections 2024-04-15T15:25:24.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:24.886+0000 7f9d9be00640 1 -- 172.21.15.150:0/2315735834 wait complete. 2024-04-15T15:25:24.907 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:25:24.908 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:25:24.908 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-04-15T15:25:24.908 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-15T15:25:24.908 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:24.908 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:25:24.908 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:24.908 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:25:24.908 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:25:24.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:24.908 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42490sec preferred_lft 42490sec 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 3e:e8:cf:d9:d1:9d brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::1086:faff:fe4c:731a/64 scope link 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 76:7b:44:2c:2d:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:25:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::747b:44ff:fe2c:2d50/64 scope link 2024-04-15T15:25:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:24.961 INFO:teuthology.orchestra.run.smithi150.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 3e:e8:cf:d9:d1:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:25:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::3ce8:cfff:fed9:d19d/64 scope link 2024-04-15T15:25:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:24.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:24.961 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:25:25.047 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-15T15:25:25.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:25.051 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-04-15T15:25:25.102 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-04-15T15:25:25.103 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:25:25.103 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:25:25.103 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:25:25.103 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:25:25.103 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:25:25.103 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:25:25.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:25:25.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:25:25.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:25:25.103 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:25:25.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:25:25.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:25.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:25.342 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:25:25.391 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:25:25.391 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:25:25.391 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:25:25.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:25.391 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-04-15T15:25:25.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:25.480 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:25.535 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-15T15:25:25.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:25.535 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:25.583 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:25.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:25.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:25:25.722 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:25:25.723 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:25:25.723 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:25.745 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:25:25.745 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:25:25.745 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:25.745 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:25.745 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:25.772 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-15T15:25:25.772 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-15T15:25:25.772 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:25.772 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:25.772 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:25.772 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:25:25.773 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:25.773 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:25:25.775 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-15T15:25:25.775 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:25:25.775 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:25:25.775 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:25:25.776 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:25:25.781 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:25:25.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:25.781 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:25.877 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:25:25.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:25:26.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:26.009 INFO:teuthology.orchestra.run.smithi150.stdout:client4888 2024-04-15T15:25:26.010 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:25:26.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.218+0000 7fdd53200640 1 Processor -- start 2024-04-15T15:25:26.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.218+0000 7fdd53200640 1 -- start start 2024-04-15T15:25:26.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.220+0000 7fdd53200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd540cee00 0x7fdd5406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:26.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.220+0000 7fdd53200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 0x7fdd540723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:26.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.220+0000 7fdd53200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5415dbb0 con 0x7fdd5415cbe0 2024-04-15T15:25:26.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.220+0000 7fdd53200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fdd5406f4f0 con 0x7fdd540cee00 2024-04-15T15:25:26.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.220+0000 7fdd53200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdd540728f0 con 0x7fdd54071f50 2024-04-15T15:25:26.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.220+0000 7fdd51e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd540cee00 0x7fdd5406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:26.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.220+0000 7fdd51e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd540cee00 0x7fdd5406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56208/0 (socket says 172.21.15.150:56208) 2024-04-15T15:25:26.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.220+0000 7fdd51e00640 1 -- 172.21.15.150:0/1279885086 learned_addr learned my addr 172.21.15.150:0/1279885086 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:26.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.221+0000 7fdd51e00640 1 -- 172.21.15.150:0/1279885086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 msgr2=0x7fdd540723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:26.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.221+0000 7fdd51e00640 1 --2- 172.21.15.150:0/1279885086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 0x7fdd540723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.221+0000 7fdd51e00640 1 -- 172.21.15.150:0/1279885086 >> v1:172.21.15.187:6789/0 conn(0x7fdd5415cbe0 legacy=0x7fdd5415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:26.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.221+0000 7fdd51e00640 1 -- 172.21.15.150:0/1279885086 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5415d8d0 con 0x7fdd540cee00 2024-04-15T15:25:26.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.222+0000 7fdd51e00640 1 --2- 172.21.15.150:0/1279885086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd540cee00 0x7fdd5406e8c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdd440098a0 tx=0x7fdd4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54741982d29ec618 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:26.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.222+0000 7fdd50a00640 1 -- 172.21.15.150:0/1279885086 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44035020 con 0x7fdd540cee00 2024-04-15T15:25:26.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.222+0000 7fdd50a00640 1 -- 172.21.15.150:0/1279885086 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd44031640 con 0x7fdd540cee00 2024-04-15T15:25:26.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.222+0000 7fdd50a00640 1 -- 172.21.15.150:0/1279885086 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4402d040 con 0x7fdd540cee00 2024-04-15T15:25:26.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.223+0000 7fdd53200640 1 -- 172.21.15.150:0/1279885086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd540cee00 msgr2=0x7fdd5406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:26.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.223+0000 7fdd53200640 1 --2- 172.21.15.150:0/1279885086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd540cee00 0x7fdd5406e8c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdd440098a0 tx=0x7fdd4402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.223+0000 7fdd53200640 1 -- 172.21.15.150:0/1279885086 shutdown_connections 2024-04-15T15:25:26.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.223+0000 7fdd53200640 1 --2- 172.21.15.150:0/1279885086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 0x7fdd540723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.223+0000 7fdd53200640 1 --2- 172.21.15.150:0/1279885086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd540cee00 0x7fdd5406e8c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.223+0000 7fdd53200640 1 -- 172.21.15.150:0/1279885086 >> 172.21.15.150:0/1279885086 conn(0x7fdd5406b930 msgr2=0x7fdd5406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:26.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.223+0000 7fdd53200640 1 -- 172.21.15.150:0/1279885086 shutdown_connections 2024-04-15T15:25:26.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.223+0000 7fdd53200640 1 -- 172.21.15.150:0/1279885086 wait complete. 2024-04-15T15:25:26.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.225+0000 7fdd53200640 1 Processor -- start 2024-04-15T15:25:26.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.225+0000 7fdd53200640 1 -- start start 2024-04-15T15:25:26.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd53200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 0x7fdd541e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:26.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd53200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd540cee00 0x7fdd5415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:26.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd53200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5415cbe0 0x7fdd541598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:26.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd53200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5406e3a0 con 0x7fdd5415cbe0 2024-04-15T15:25:26.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd53200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5406e0c0 con 0x7fdd540cee00 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd53200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdd5415dda0 con 0x7fdd54071f50 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd52800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5415cbe0 0x7fdd541598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd52800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5415cbe0 0x7fdd541598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:56220/0 (socket says 172.21.15.150:56220) 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd52800640 1 -- 172.21.15.150:0/2150965153 learned_addr learned my addr 172.21.15.150:0/2150965153 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd51e00640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 0x7fdd541e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.226+0000 7fdd51400640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd540cee00 0x7fdd5415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd52800640 1 -- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 msgr2=0x7fdd541e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd52800640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 0x7fdd541e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd52800640 1 -- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd540cee00 msgr2=0x7fdd5415ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd52800640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd540cee00 0x7fdd5415ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd52800640 1 -- 172.21.15.150:0/2150965153 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd541e78e0 con 0x7fdd5415cbe0 2024-04-15T15:25:26.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd51400640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd540cee00 0x7fdd5415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:26.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd52800640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5415cbe0 0x7fdd541598f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00b970 tx=0x7fdd4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:26.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd4a000640 1 -- 172.21.15.150:0/2150965153 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c019070 con 0x7fdd5415cbe0 2024-04-15T15:25:26.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd4a000640 1 -- 172.21.15.150:0/2150965153 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00cb40 con 0x7fdd5415cbe0 2024-04-15T15:25:26.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd4a000640 1 -- 172.21.15.150:0/2150965153 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c014440 con 0x7fdd5415cbe0 2024-04-15T15:25:26.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541e6320 con 0x7fdd5415cbe0 2024-04-15T15:25:26.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.227+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541e6ba0 con 0x7fdd5415cbe0 2024-04-15T15:25:26.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.229+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd540755c0 con 0x7fdd5415cbe0 2024-04-15T15:25:26.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.229+0000 7fdd4a000640 1 -- 172.21.15.150:0/2150965153 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdd4c01c070 con 0x7fdd5415cbe0 2024-04-15T15:25:26.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.230+0000 7fdd4a000640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdd3006c980 0x7fdd3006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:26.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.230+0000 7fdd4a000640 1 -- 172.21.15.150:0/2150965153 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdd4c08b800 con 0x7fdd5415cbe0 2024-04-15T15:25:26.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.231+0000 7fdd51e00640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdd3006c980 0x7fdd3006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:26.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.232+0000 7fdd51e00640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdd3006c980 0x7fdd3006ee40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd44036000 tx=0x7fdd4403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:26.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.235+0000 7fdd4a000640 1 -- 172.21.15.150:0/2150965153 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd4c059bd0 con 0x7fdd5415cbe0 2024-04-15T15:25:26.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.476+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdd5406c090 con 0x7fdd5415cbe0 2024-04-15T15:25:26.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.477+0000 7fdd4a000640 1 -- 172.21.15.150:0/2150965153 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fdd4c0594d0 con 0x7fdd5415cbe0 2024-04-15T15:25:26.479 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:25:26.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdd3006c980 msgr2=0x7fdd3006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdd3006c980 0x7fdd3006ee40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd44036000 tx=0x7fdd4403c040 comp rx=0 tx=0).stop 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5415cbe0 msgr2=0x7fdd541598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5415cbe0 0x7fdd541598f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00b970 tx=0x7fdd4c00be30 comp rx=0 tx=0).stop 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 shutdown_connections 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5415cbe0 0x7fdd541598f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd540cee00 0x7fdd5415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fdd3006c980 0x7fdd3006ee40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 --2- 172.21.15.150:0/2150965153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd54071f50 0x7fdd541e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:26.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 >> 172.21.15.150:0/2150965153 conn(0x7fdd5406b930 msgr2=0x7fdd54160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:26.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.480+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 shutdown_connections 2024-04-15T15:25:26.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:26.481+0000 7fdd53200640 1 -- 172.21.15.150:0/2150965153 wait complete. 2024-04-15T15:25:26.502 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:25:26.502 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:25:26.502 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-04-15T15:25:26.503 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-15T15:25:26.503 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:26.503 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:25:26.503 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:26.503 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:25:26.503 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:25:26.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:26.503 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-04-15T15:25:26.563 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42536sec preferred_lft 42536sec 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 16:44:cc:5d:db:12 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:26.564 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::442c:2bff:fe33:cf6c/64 scope link 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 5a:c2:59:f5:7f:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::58c2:59ff:fef5:7f4e/64 scope link 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 16:44:cc:5d:db:12 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::1444:ccff:fe5d:db12/64 scope link 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:26.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:26.566 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:25:26.656 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-15T15:25:26.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:26.661 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-04-15T15:25:26.714 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-04-15T15:25:26.715 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:25:26.715 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:25:26.715 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:25:26.715 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:25:26.716 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:25:26.716 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:25:26.716 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:25:26.716 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:25:26.716 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:25:26.716 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:25:26.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:25:26.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:26.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:26.952 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:25:26.998 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:25:26.999 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:25:26.999 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:25:26.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:26.999 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-04-15T15:25:27.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:27.084 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:27.137 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-15T15:25:27.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:27.138 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:27.184 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:27.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:27.184 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:25:27.366 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:25:27.366 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:25:27.366 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:27.387 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:25:27.387 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:25:27.388 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:27.388 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:27.388 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:27.419 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-15T15:25:27.419 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-15T15:25:27.420 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:27.420 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:27.420 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:27.420 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:25:27.420 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:27.420 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:25:27.422 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-15T15:25:27.422 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:25:27.422 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:25:27.422 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:25:27.422 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:25:27.428 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:25:27.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:27.428 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:27.513 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:25:27.617 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:25:27.623 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:27.632 INFO:teuthology.orchestra.run.smithi187.stdout:client5424 2024-04-15T15:25:27.632 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:25:27.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.848+0000 7f3012e00640 1 Processor -- start 2024-04-15T15:25:27.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.848+0000 7f3012e00640 1 -- start start 2024-04-15T15:25:27.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.849+0000 7f3012e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f300c15b860 0x7f300c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:27.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.849+0000 7f3012e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300c0cdfb0 con 0x7f300c06f320 2024-04-15T15:25:27.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.849+0000 7f3012e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300c15f2b0 con 0x7f300c15b480 2024-04-15T15:25:27.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.849+0000 7f3012e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f300c15f060 con 0x7f300c15b860 2024-04-15T15:25:27.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.850+0000 7f3012400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f300c06f320 0x7f300c0b3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:43062/0 (socket says 172.21.15.150:43062) 2024-04-15T15:25:27.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.850+0000 7f3012400640 1 -- 172.21.15.150:0/2145118344 learned_addr learned my addr 172.21.15.150:0/2145118344 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:27.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.850+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175678589 0 0) 0x7f300c0cdfb0 con 0x7f300c06f320 2024-04-15T15:25:27.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.850+0000 7f3011000640 1 --2- 172.21.15.150:0/2145118344 >> v2:172.21.15.187:3300/0 conn(0x7f300c15b860 0x7f300c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:27.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.850+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ff4003520 con 0x7f300c06f320 2024-04-15T15:25:27.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 545026788 0 0) 0x7f2ff4003520 con 0x7f300c06f320 2024-04-15T15:25:27.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 >> v2:172.21.15.187:3300/0 conn(0x7f300c15b860 msgr2=0x7f300c06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:27.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 --2- 172.21.15.150:0/2145118344 >> v2:172.21.15.187:3300/0 conn(0x7f300c15b860 0x7f300c06ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 >> v1:172.21.15.187:6789/0 conn(0x7f300c15b480 legacy=0x7f300c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c15f590 con 0x7f300c06f320 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f30040019f0 con 0x7f300c06f320 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 >> v1:172.21.15.150:6789/0 conn(0x7f300c06f320 legacy=0x7f300c0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff4004800 0x7f2ff4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff40051c0 0x7f2ff400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f300c15f060 con 0x7f2ff40039b0 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff400c110 con 0x7f2ff4004800 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff400c310 con 0x7f2ff40051c0 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.851+0000 7f3011a00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.852+0000 7f3011a00640 1 -- 172.21.15.150:0/2145118344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff40051c0 msgr2=0x7f2ff400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:27.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.852+0000 7f3011a00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff40051c0 0x7f2ff400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:27.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.852+0000 7f3011a00640 1 -- 172.21.15.150:0/2145118344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff4004800 msgr2=0x7f2ff4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:27.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.852+0000 7f3011a00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff4004800 0x7f2ff4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:27.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.852+0000 7f3011a00640 1 -- 172.21.15.150:0/2145118344 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff4003520 con 0x7f2ff40039b0 2024-04-15T15:25:27.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.852+0000 7f3012400640 1 -- 172.21.15.150:0/2145118344 reap_dead start 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.853+0000 7f3011a00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003dd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3000001750 tx=0x7f2ff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9704ba4492b14bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.853+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000001370 con 0x7f2ff40039b0 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.853+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30000052d0 con 0x7f2ff40039b0 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.853+0000 7f300be00640 1 -- 172.21.15.150:0/2145118344 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000003830 con 0x7f2ff40039b0 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.854+0000 7f3012e00640 1 -- 172.21.15.150:0/2145118344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff40039b0 msgr2=0x7f2ff4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.854+0000 7f3012e00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003dd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3000001750 tx=0x7f2ff4005830 comp rx=0 tx=0).stop 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.854+0000 7f3012e00640 1 -- 172.21.15.150:0/2145118344 shutdown_connections 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.854+0000 7f3012e00640 1 --2- 172.21.15.150:0/2145118344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff40039b0 0x7f2ff4003dd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.854+0000 7f3012e00640 1 -- 172.21.15.150:0/2145118344 >> 172.21.15.150:0/2145118344 conn(0x7f300c06b930 msgr2=0x7f300c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.854+0000 7f3012e00640 1 -- 172.21.15.150:0/2145118344 shutdown_connections 2024-04-15T15:25:27.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.854+0000 7f3012e00640 1 -- 172.21.15.150:0/2145118344 wait complete. 2024-04-15T15:25:27.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.856+0000 7f3012e00640 1 Processor -- start 2024-04-15T15:25:27.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.856+0000 7f3012e00640 1 -- start start 2024-04-15T15:25:27.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3012e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff40039b0 0x7f300c1f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:27.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3012e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f300c147160 0x7f300c1f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:27.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3012e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f300c0b1c70 0x7f300c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:27.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3012e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f300c15fb50 con 0x7f300c147160 2024-04-15T15:25:27.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3012e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f300c15fec0 con 0x7f2ff40039b0 2024-04-15T15:25:27.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3012e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f300c0cd550 con 0x7f300c0b1c70 2024-04-15T15:25:27.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3011000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f300c147160 0x7f300c1f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:27.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3011000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f300c147160 0x7f300c1f8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55628/0 (socket says 172.21.15.150:55628) 2024-04-15T15:25:27.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3011000640 1 -- 172.21.15.150:0/1902931683 learned_addr learned my addr 172.21.15.150:0/1902931683 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:27.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3012400640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f300c0b1c70 0x7f300c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:27.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3011a00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff40039b0 0x7f300c1f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:27.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3011000640 1 -- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f300c0b1c70 msgr2=0x7f300c154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3011000640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f300c0b1c70 0x7f300c154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3011000640 1 -- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff40039b0 msgr2=0x7f300c1f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.857+0000 7f3011000640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff40039b0 0x7f300c1f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.858+0000 7f3011000640 1 -- 172.21.15.150:0/1902931683 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c143e10 con 0x7f300c147160 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.858+0000 7f3011a00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff40039b0 0x7f300c1f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.858+0000 7f3011000640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f300c147160 0x7f300c1f8230 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800d560 tx=0x7f2ff800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.858+0000 7f3009600640 1 -- 172.21.15.150:0/1902931683 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80125f0 con 0x7f300c147160 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.858+0000 7f3009600640 1 -- 172.21.15.150:0/1902931683 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8019070 con 0x7f300c147160 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.858+0000 7f3009600640 1 -- 172.21.15.150:0/1902931683 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80129d0 con 0x7f300c147160 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.858+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c1440f0 con 0x7f300c147160 2024-04-15T15:25:27.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.858+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1446b0 con 0x7f300c147160 2024-04-15T15:25:27.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.859+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c0b3930 con 0x7f300c147160 2024-04-15T15:25:27.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.860+0000 7f3009600640 1 -- 172.21.15.150:0/1902931683 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f2ff8012b70 con 0x7f300c147160 2024-04-15T15:25:27.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.861+0000 7f3009600640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2fec06c980 0x7f2fec06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:27.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.861+0000 7f3009600640 1 -- 172.21.15.150:0/1902931683 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2ff808bed0 con 0x7f300c147160 2024-04-15T15:25:27.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.862+0000 7f3011a00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2fec06c980 0x7f2fec06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:27.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.863+0000 7f3011a00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2fec06c980 0x7f2fec06ee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3000033000 tx=0x7f3000009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:27.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:27.866+0000 7f3009600640 1 -- 172.21.15.150:0/1902931683 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ff805a2a0 con 0x7f300c147160 2024-04-15T15:25:28.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.114+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f300c0755a0 con 0x7f300c147160 2024-04-15T15:25:28.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.115+0000 7f3009600640 1 -- 172.21.15.150:0/1902931683 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f2ff8059ba0 con 0x7f300c147160 2024-04-15T15:25:28.117 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:25:28.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2fec06c980 msgr2=0x7f2fec06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:28.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2fec06c980 0x7f2fec06ee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3000033000 tx=0x7f3000009030 comp rx=0 tx=0).stop 2024-04-15T15:25:28.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f300c147160 msgr2=0x7f300c1f8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:28.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f300c147160 0x7f300c1f8230 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800d560 tx=0x7f2ff800da20 comp rx=0 tx=0).stop 2024-04-15T15:25:28.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 shutdown_connections 2024-04-15T15:25:28.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f2fec06c980 0x7f2fec06ee40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:28.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f300c0b1c70 0x7f300c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:28.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f300c147160 0x7f300c1f8230 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:28.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 --2- 172.21.15.150:0/1902931683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff40039b0 0x7f300c1f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:28.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 >> 172.21.15.150:0/1902931683 conn(0x7f300c06b930 msgr2=0x7f300c0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:28.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.117+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 shutdown_connections 2024-04-15T15:25:28.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28.118+0000 7f3012e00640 1 -- 172.21.15.150:0/1902931683 wait complete. 2024-04-15T15:25:28.138 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:25:28.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:25:28.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-04-15T15:25:28.138 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-15T15:25:28.138 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-15T15:25:28.138 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:25:28.138 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:28.138 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:25:28.139 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:25:28.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:28.139 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-04-15T15:25:28.188 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:25:28.188 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:25:28.188 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:25:28.188 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:28.188 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-15T15:25:28.188 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42487sec preferred_lft 42487sec 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2024-04-15T15:25:28.189 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 3e:e8:cf:d9:d1:9d brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::1086:faff:fe4c:731a/64 scope link 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 76:7b:44:2c:2d:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::747b:44ff:fe2c:2d50/64 scope link 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 3e:e8:cf:d9:d1:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::3ce8:cfff:fed9:d19d/64 scope link 2024-04-15T15:25:28.190 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:28.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:28.191 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:25:28.277 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-15T15:25:28.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:28.281 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-04-15T15:25:28.332 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-04-15T15:25:28.332 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:25:28.332 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:25:28.332 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:25:28.332 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:25:28.333 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:25:28.333 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-15T15:25:28.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:25:28.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:25:28.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:25:28.334 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-15T15:25:28.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:25:28.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:28.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:28.581 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:25:28.627 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:25:28.627 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:25:28.628 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:25:28.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:28.628 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.1 up 2024-04-15T15:25:28.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:28.765 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-15T15:25:28.818 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-15T15:25:28.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:28.818 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:25:28.867 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:28.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:28.867 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:25:29.000 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:25:29.000 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:25:29.001 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:29.022 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:25:29.022 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:25:29.022 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:29.022 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:29.022 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:29.050 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-15T15:25:29.050 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-15T15:25:29.050 INFO:teuthology.orchestra.run.smithi150.stdout:invalid new device string format 2024-04-15T15:25:29.050 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:29.050 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:29.050 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:25:29.050 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:29.050 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:25:29.052 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-15T15:25:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:25:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:25:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:25:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:25:29.056 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:25:29.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:29.056 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-15T15:25:29.135 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:25:29.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:25:29.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:29.253 INFO:teuthology.orchestra.run.smithi150.stdout:client4673 2024-04-15T15:25:29.253 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-15T15:25:29.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.457+0000 7fa8d3200640 1 Processor -- start 2024-04-15T15:25:29.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.457+0000 7fa8d3200640 1 -- start start 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d3200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa8d415d2f0 0x7fa8d406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d3200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d415dbb0 con 0x7fa8d40cee00 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d3200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d415d8d0 con 0x7fa8d4072660 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d3200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa8d406f4f0 con 0x7fa8d415d2f0 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d2800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa8d415d2f0 0x7fa8d406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d2800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa8d415d2f0 0x7fa8d406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55636/0 (socket says 172.21.15.150:55636) 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d2800640 1 -- 172.21.15.150:0/1521248682 learned_addr learned my addr 172.21.15.150:0/1521248682 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d1e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa8d40cee00 0x7fa8d4071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:43076/0 (socket says 172.21.15.150:43076) 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d2800640 1 -- 172.21.15.150:0/1521248682 >> v1:172.21.15.187:6789/0 conn(0x7fa8d4072660 legacy=0x7fa8d415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:29.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d2800640 1 -- 172.21.15.150:0/1521248682 >> v1:172.21.15.150:6789/0 conn(0x7fa8d40cee00 legacy=0x7fa8d4071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:29.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.459+0000 7fa8d2800640 1 -- 172.21.15.150:0/1521248682 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d406dde0 con 0x7fa8d415d2f0 2024-04-15T15:25:29.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.460+0000 7fa8d2800640 1 --2- 172.21.15.150:0/1521248682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415d2f0 0x7fa8d406e8c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009870 tx=0x7fa8cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b044cacaa7ab0008 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:29.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.461+0000 7fa8d1400640 1 -- 172.21.15.150:0/1521248682 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc034070 con 0x7fa8d415d2f0 2024-04-15T15:25:29.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.461+0000 7fa8d1400640 1 -- 172.21.15.150:0/1521248682 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8cc030860 con 0x7fa8d415d2f0 2024-04-15T15:25:29.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.461+0000 7fa8d1400640 1 -- 172.21.15.150:0/1521248682 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc030c30 con 0x7fa8d415d2f0 2024-04-15T15:25:29.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.461+0000 7fa8d3200640 1 -- 172.21.15.150:0/1521248682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415d2f0 msgr2=0x7fa8d406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:29.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.461+0000 7fa8d3200640 1 --2- 172.21.15.150:0/1521248682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415d2f0 0x7fa8d406e8c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009870 tx=0x7fa8cc02b9a0 comp rx=0 tx=0).stop 2024-04-15T15:25:29.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.462+0000 7fa8d3200640 1 -- 172.21.15.150:0/1521248682 shutdown_connections 2024-04-15T15:25:29.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.462+0000 7fa8d3200640 1 --2- 172.21.15.150:0/1521248682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415d2f0 0x7fa8d406e8c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:29.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.462+0000 7fa8d3200640 1 -- 172.21.15.150:0/1521248682 >> 172.21.15.150:0/1521248682 conn(0x7fa8d406b930 msgr2=0x7fa8d406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:29.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.462+0000 7fa8d3200640 1 -- 172.21.15.150:0/1521248682 shutdown_connections 2024-04-15T15:25:29.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.462+0000 7fa8d3200640 1 -- 172.21.15.150:0/1521248682 wait complete. 2024-04-15T15:25:29.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.463+0000 7fa8d3200640 1 Processor -- start 2024-04-15T15:25:29.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.464+0000 7fa8d3200640 1 -- start start 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d4072660 0x7fa8d414ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d40cee00 0x7fa8d41f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d415d2f0 0x7fa8d40b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d415dda0 con 0x7fa8d4072660 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d406e3a0 con 0x7fa8d40cee00 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d415df20 con 0x7fa8d415d2f0 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d4072660 0x7fa8d414ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d4072660 0x7fa8d414ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55642/0 (socket says 172.21.15.150:55642) 2024-04-15T15:25:29.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d1e00640 1 -- 172.21.15.150:0/2078222385 learned_addr learned my addr 172.21.15.150:0/2078222385 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8c9400640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d40cee00 0x7fa8d41f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d1e00640 1 -- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d415d2f0 msgr2=0x7fa8d40b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d1e00640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d415d2f0 0x7fa8d40b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d1e00640 1 -- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d40cee00 msgr2=0x7fa8d41f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d1e00640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d40cee00 0x7fa8d41f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8d1e00640 1 -- 172.21.15.150:0/2078222385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d41f9040 con 0x7fa8d4072660 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.465+0000 7fa8c9400640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d40cee00 0x7fa8d41f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.466+0000 7fa8d1e00640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d4072660 0x7fa8d414ee40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c790 tx=0x7fa8c400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.466+0000 7fa8caa00640 1 -- 172.21.15.150:0/2078222385 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4017070 con 0x7fa8d4072660 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.466+0000 7fa8caa00640 1 -- 172.21.15.150:0/2078222385 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c400a640 con 0x7fa8d4072660 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.466+0000 7fa8caa00640 1 -- 172.21.15.150:0/2078222385 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400a9f0 con 0x7fa8d4072660 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.466+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d41f9320 con 0x7fa8d4072660 2024-04-15T15:25:29.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.466+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d414f380 con 0x7fa8d4072660 2024-04-15T15:25:29.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.467+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d40755c0 con 0x7fa8d4072660 2024-04-15T15:25:29.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.468+0000 7fa8caa00640 1 -- 172.21.15.150:0/2078222385 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa8c401a070 con 0x7fa8d4072660 2024-04-15T15:25:29.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.469+0000 7fa8caa00640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8b006c980 0x7fa8b006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:29.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.469+0000 7fa8caa00640 1 -- 172.21.15.150:0/2078222385 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa8c408b0d0 con 0x7fa8d4072660 2024-04-15T15:25:29.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.470+0000 7fa8c9400640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8b006c980 0x7fa8b006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:29.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.471+0000 7fa8c9400640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8b006c980 0x7fa8b006ee40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4155670 tx=0x7fa8bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:29.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.474+0000 7fa8caa00640 1 -- 172.21.15.150:0/2078222385 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8c4058f50 con 0x7fa8d4072660 2024-04-15T15:25:29.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.715+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa8d406cf30 con 0x7fa8d4072660 2024-04-15T15:25:29.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.716+0000 7fa8caa00640 1 -- 172.21.15.150:0/2078222385 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fa8c405d250 con 0x7fa8d4072660 2024-04-15T15:25:29.718 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-15T15:25:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8b006c980 msgr2=0x7fa8b006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8b006c980 0x7fa8b006ee40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4155670 tx=0x7fa8bc009040 comp rx=0 tx=0).stop 2024-04-15T15:25:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d4072660 msgr2=0x7fa8d414ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d4072660 0x7fa8d414ee40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c790 tx=0x7fa8c400cc50 comp rx=0 tx=0).stop 2024-04-15T15:25:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 shutdown_connections 2024-04-15T15:25:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8b006c980 0x7fa8b006ee40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d415d2f0 0x7fa8d40b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:29.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d40cee00 0x7fa8d41f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 --2- 172.21.15.150:0/2078222385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d4072660 0x7fa8d414ee40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 >> 172.21.15.150:0/2078222385 conn(0x7fa8d406b930 msgr2=0x7fa8d40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 shutdown_connections 2024-04-15T15:25:29.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:29.719+0000 7fa8d3200640 1 -- 172.21.15.150:0/2078222385 wait complete. 2024-04-15T15:25:29.741 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-15T15:25:29.741 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-15T15:25:29.741 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-04-15T15:25:29.742 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-15T15:25:29.742 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-15T15:25:29.742 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-15T15:25:29.742 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:29.742 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-15T15:25:29.742 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:25:29.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:29.742 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-04-15T15:25:29.799 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-15T15:25:29.800 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42533sec preferred_lft 42533sec 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 16:44:cc:5d:db:12 brd ff:ff:ff:ff:ff:ff 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::442c:2bff:fe33:cf6c/64 scope link 2024-04-15T15:25:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:29.802 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:29.802 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 5a:c2:59:f5:7f:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:25:29.802 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::58c2:59ff:fef5:7f4e/64 scope link 2024-04-15T15:25:29.802 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:29.802 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-15T15:25:29.802 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 16:44:cc:5d:db:12 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:25:29.802 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::1444:ccff:fe5d:db12/64 scope link 2024-04-15T15:25:29.802 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-04-15T15:25:29.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:29.803 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-15T15:25:29.893 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-15T15:25:29.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:29.897 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-04-15T15:25:29.947 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-04-15T15:25:29.947 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-15T15:25:29.947 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-15T15:25:29.948 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-15T15:25:29.948 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-15T15:25:29.949 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-15T15:25:29.949 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-04-15T15:25:29.949 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-15T15:25:29.949 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-15T15:25:29.949 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-15T15:25:29.949 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-04-15T15:25:30.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-15T15:25:30.186 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:30.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:30.197 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:25:30.244 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:25:30.244 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:25:30.245 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:25:30.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:30.245 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.1 up 2024-04-15T15:25:30.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:30.335 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-15T15:25:30.391 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-15T15:25:30.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-15T15:25:30.391 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:25:30.442 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:30.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:30.442 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:25:30.583 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:25:30.583 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:25:30.583 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:30.603 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:25:30.604 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:25:30.604 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:30.604 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:30.604 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:30.633 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-15T15:25:30.633 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-15T15:25:30.633 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-04-15T15:25:30.633 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-04-15T15:25:30.634 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-15T15:25:30.634 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-04-15T15:25:30.634 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:6789,172.21.15.187:6789,172.21.15.187:6790:/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 2024-04-15T15:25:30.634 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4" will pass to kernel 2024-04-15T15:25:30.638 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-15T15:25:30.638 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-04-15T15:25:30.638 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-15T15:25:30.638 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-04-15T15:25:30.638 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-15T15:25:30.642 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-15T15:25:30.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:30.642 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-15T15:25:30.724 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-15T15:25:30.828 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:25:30.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-15T15:25:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:30.842 INFO:teuthology.orchestra.run.smithi187.stdout:client4676 2024-04-15T15:25:30.843 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'touch file') 2024-04-15T15:25:30.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-15T15:25:30.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-15T15:25:30.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:30.993 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quiesce.block -v 1 file) 2024-04-15T15:25:31.100 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 quiesce path /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b --wait 2024-04-15T15:25:31.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.321+0000 7fb409400640 1 Processor -- start 2024-04-15T15:25:31.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.322+0000 7fb409400640 1 -- start start 2024-04-15T15:25:31.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.323+0000 7fb409400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb4040cfa80 0x7fb4040b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.323+0000 7fb409400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb4040b0b90 0x7fb40415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.323+0000 7fb409400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4040b0fc0 con 0x7fb4041460f0 2024-04-15T15:25:31.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.323+0000 7fb409400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fb404070e90 con 0x7fb4040b0b90 2024-04-15T15:25:31.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.323+0000 7fb409400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb4040cff50 con 0x7fb4040cfa80 2024-04-15T15:25:31.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.323+0000 7fb403e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fb4041460f0 0x7fb4040cf370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:43080/0 (socket says 172.21.15.150:43080) 2024-04-15T15:25:31.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.323+0000 7fb403e00640 1 -- 172.21.15.150:0/1160669486 learned_addr learned my addr 172.21.15.150:0/1160669486 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:31.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.323+0000 7fb408a00640 1 --2- 172.21.15.150:0/1160669486 >> v2:172.21.15.150:3300/0 conn(0x7fb4040b0b90 0x7fb40415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.324+0000 7fb403400640 1 --2- 172.21.15.150:0/1160669486 >> v2:172.21.15.187:3300/0 conn(0x7fb4040cfa80 0x7fb4040b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.324+0000 7fb402a00640 1 -- 172.21.15.150:0/1160669486 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506773235 0 0) 0x7fb4040b0fc0 con 0x7fb4041460f0 2024-04-15T15:25:31.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.324+0000 7fb402a00640 1 -- 172.21.15.150:0/1160669486 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3e8003520 con 0x7fb4041460f0 2024-04-15T15:25:31.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.324+0000 7fb408a00640 1 -- 172.21.15.150:0/1160669486 >> v2:172.21.15.187:3300/0 conn(0x7fb4040cfa80 msgr2=0x7fb4040b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.324+0000 7fb408a00640 1 --2- 172.21.15.150:0/1160669486 >> v2:172.21.15.187:3300/0 conn(0x7fb4040cfa80 0x7fb4040b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.324+0000 7fb408a00640 1 -- 172.21.15.150:0/1160669486 >> v1:172.21.15.150:6789/0 conn(0x7fb4041460f0 legacy=0x7fb4040cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.324+0000 7fb408a00640 1 -- 172.21.15.150:0/1160669486 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4040ce0c0 con 0x7fb4040b0b90 2024-04-15T15:25:31.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.325+0000 7fb408a00640 1 --2- 172.21.15.150:0/1160669486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040b0b90 0x7fb40415cb10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb4040b12b0 tx=0x7fb3f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1598c5ee6b729a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.325+0000 7fb402a00640 1 -- 172.21.15.150:0/1160669486 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40315a0 con 0x7fb4040b0b90 2024-04-15T15:25:31.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.325+0000 7fb402a00640 1 -- 172.21.15.150:0/1160669486 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f4037070 con 0x7fb4040b0b90 2024-04-15T15:25:31.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.325+0000 7fb402a00640 1 -- 172.21.15.150:0/1160669486 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4031960 con 0x7fb4040b0b90 2024-04-15T15:25:31.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.326+0000 7fb409400640 1 -- 172.21.15.150:0/1160669486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040b0b90 msgr2=0x7fb40415cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.326+0000 7fb409400640 1 --2- 172.21.15.150:0/1160669486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040b0b90 0x7fb40415cb10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb4040b12b0 tx=0x7fb3f402b840 comp rx=0 tx=0).stop 2024-04-15T15:25:31.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.326+0000 7fb409400640 1 -- 172.21.15.150:0/1160669486 shutdown_connections 2024-04-15T15:25:31.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.326+0000 7fb409400640 1 --2- 172.21.15.150:0/1160669486 >> v2:172.21.15.187:3300/0 conn(0x7fb4040cfa80 0x7fb4040b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.326+0000 7fb409400640 1 --2- 172.21.15.150:0/1160669486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040b0b90 0x7fb40415cb10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.326+0000 7fb409400640 1 -- 172.21.15.150:0/1160669486 >> 172.21.15.150:0/1160669486 conn(0x7fb4040b3370 msgr2=0x7fb4040b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:31.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.326+0000 7fb409400640 1 -- 172.21.15.150:0/1160669486 shutdown_connections 2024-04-15T15:25:31.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.326+0000 7fb409400640 1 -- 172.21.15.150:0/1160669486 wait complete. 2024-04-15T15:25:31.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.328+0000 7fb409400640 1 Processor -- start 2024-04-15T15:25:31.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.328+0000 7fb409400640 1 -- start start 2024-04-15T15:25:31.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb409400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b0b90 0x7fb40415e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb409400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040cfa80 0x7fb40414c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb409400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4041460f0 0x7fb40413cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb409400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040ce960 con 0x7fb4040cfa80 2024-04-15T15:25:31.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb409400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040ce680 con 0x7fb4041460f0 2024-04-15T15:25:31.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb409400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb4040cee80 con 0x7fb4040b0b90 2024-04-15T15:25:31.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb403400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040cfa80 0x7fb40414c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb403400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040cfa80 0x7fb40414c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55664/0 (socket says 172.21.15.150:55664) 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.329+0000 7fb403400640 1 -- 172.21.15.150:0/1317081483 learned_addr learned my addr 172.21.15.150:0/1317081483 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb408a00640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4041460f0 0x7fb40413cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb403e00640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b0b90 0x7fb40415e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb403400640 1 -- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b0b90 msgr2=0x7fb40415e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb403400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b0b90 0x7fb40415e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb403400640 1 -- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4041460f0 msgr2=0x7fb40413cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb403400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4041460f0 0x7fb40413cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb403400640 1 -- 172.21.15.150:0/1317081483 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40413bc80 con 0x7fb4040cfa80 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb408a00640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4041460f0 0x7fb40413cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb403400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040cfa80 0x7fb40414c8f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec007ef0 tx=0x7fb3ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb3fbe00640 1 -- 172.21.15.150:0/1317081483 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec014620 con 0x7fb4040cfa80 2024-04-15T15:25:31.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb3fbe00640 1 -- 172.21.15.150:0/1317081483 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ec019070 con 0x7fb4040cfa80 2024-04-15T15:25:31.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.330+0000 7fb409400640 1 -- 172.21.15.150:0/1317081483 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40413bf60 con 0x7fb4040cfa80 2024-04-15T15:25:31.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.331+0000 7fb3fbe00640 1 -- 172.21.15.150:0/1317081483 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec014a00 con 0x7fb4040cfa80 2024-04-15T15:25:31.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.331+0000 7fb409400640 1 -- 172.21.15.150:0/1317081483 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40413c800 con 0x7fb4040cfa80 2024-04-15T15:25:31.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.333+0000 7fb3fbe00640 1 -- 172.21.15.150:0/1317081483 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb3ec014ba0 con 0x7fb4040cfa80 2024-04-15T15:25:31.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.334+0000 7fb3fbe00640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3e006c980 0x7fb3e006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.334+0000 7fb3fbe00640 1 -- 172.21.15.150:0/1317081483 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb3ec08c660 con 0x7fb4040cfa80 2024-04-15T15:25:31.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.334+0000 7fb40d5f8740 1 Processor -- start 2024-04-15T15:25:31.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.334+0000 7fb40d5f8740 1 -- start start 2024-04-15T15:25:31.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.334+0000 7fb403e00640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3e006c980 0x7fb3e006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.335+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb1524860 0x559fb14f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.335+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 0x559fb15275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.335+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb14f7d40 0x559fb1527b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.335+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559fb1462540 con 0x559fb14f7d40 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.335+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559fb1408b70 con 0x559fb1524860 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.335+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559fb145e500 con 0x559fb1524c40 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.335+0000 7fb408a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb14f7d40 0x559fb1527b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb408a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb14f7d40 0x559fb1527b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55678/0 (socket says 172.21.15.150:55678) 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb408a00640 1 -- 172.21.15.150:0/1062379282 learned_addr learned my addr 172.21.15.150:0/1062379282 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb403e00640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3e006c980 0x7fb3e006ee40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005880 tx=0x7fb3fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb403400640 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 0x559fb15275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb403e00640 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb1524860 0x559fb14f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb408a00640 1 -- 172.21.15.150:0/1062379282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 msgr2=0x559fb15275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb408a00640 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 0x559fb15275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb408a00640 1 -- 172.21.15.150:0/1062379282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb1524860 msgr2=0x559fb14f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb408a00640 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb1524860 0x559fb14f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb408a00640 1 -- 172.21.15.150:0/1062379282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb13edde0 con 0x559fb14f7d40 2024-04-15T15:25:31.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.336+0000 7fb408a00640 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb14f7d40 0x559fb1527b30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009890 tx=0x7fb3f4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443cbc154dce69a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb3f3e00640 1 -- 172.21.15.150:0/1062379282 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40317a0 con 0x559fb14f7d40 2024-04-15T15:25:31.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb3f3e00640 1 -- 172.21.15.150:0/1062379282 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f4037040 con 0x559fb14f7d40 2024-04-15T15:25:31.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb3f3e00640 1 -- 172.21.15.150:0/1062379282 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4031b40 con 0x559fb14f7d40 2024-04-15T15:25:31.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb40d5f8740 1 -- 172.21.15.150:0/1062379282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb14f7d40 msgr2=0x559fb1527b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb14f7d40 0x559fb1527b30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009890 tx=0x7fb3f4030000 comp rx=0 tx=0).stop 2024-04-15T15:25:31.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb40d5f8740 1 -- 172.21.15.150:0/1062379282 shutdown_connections 2024-04-15T15:25:31.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb14f7d40 0x559fb1527b30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 0x559fb15275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/1062379282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb1524860 0x559fb14f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.337+0000 7fb40d5f8740 1 -- 172.21.15.150:0/1062379282 >> 172.21.15.150:0/1062379282 conn(0x559fb14f9a70 msgr2=0x559fb15425f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:31.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.338+0000 7fb40d5f8740 1 -- 172.21.15.150:0/1062379282 shutdown_connections 2024-04-15T15:25:31.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.338+0000 7fb40d5f8740 1 -- 172.21.15.150:0/1062379282 wait complete. 2024-04-15T15:25:31.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.338+0000 7fb40d5f8740 1 -- start start 2024-04-15T15:25:31.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.338+0000 7fb40d5f8740 1 Processor -- start 2024-04-15T15:25:31.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.339+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524860 0x559fb1553630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.339+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 0x559fb1520450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.339+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb1552440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.339+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559fb14fc010 con 0x559fb1524860 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.339+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559fb141bf40 con 0x559fb152b110 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.339+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559fb14fb400 con 0x559fb1524c40 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.339+0000 7fb40d5f8740 10 client.0 _ll_register_callbacks cb 0x559fb1405550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524860 0x559fb1553630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524860 0x559fb1553630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55690/0 (socket says 172.21.15.150:55690) 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 -- 172.21.15.150:0/2341752655 learned_addr learned my addr 172.21.15.150:0/2341752655 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb408a00640 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb1552440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 -- 172.21.15.150:0/2341752655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 msgr2=0x559fb1520450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403400640 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 0x559fb1520450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 0x559fb1520450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.344 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 -- 172.21.15.150:0/2341752655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 msgr2=0x559fb1552440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb1552440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 -- 172.21.15.150:0/2341752655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb1241030 con 0x559fb1524860 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb403e00640 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524860 0x559fb1553630 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005ef0 tx=0x7fb3fc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ebcc9ff7e177e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.340+0000 7fb3f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2341752655 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc027060 con 0x559fb1524860 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2341752655 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb3fc010040 con 0x559fb1524860 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2341752655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f4037040 con 0x559fb1524860 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2341752655 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc007560 con 0x559fb1524860 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb40d5f8740 10 client.5454 fetch_fsmap learned FSMap version 17 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2341752655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559fb1174c20 con 0x559fb1524860 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2341752655 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f4037040 con 0x559fb1524860 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2341752655 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fb3fc022430 con 0x559fb1524860 2024-04-15T15:25:31.345 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb40d5f8740 10 client.5454 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:25:31.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb40d5f8740 10 client.5454 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-15T15:25:31.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb40d5f8740 20 client.5454 populate_metadata read hostname 'smithi150' 2024-04-15T15:25:31.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.341+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x559fb1552980 0x559fb15269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.342+0000 7fb40d5f8740 4 client.5454 mds_command: new command op to 5112 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:25:31.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.342+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2341752655 --> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fb146ff60 con 0x559fb1552980 2024-04-15T15:25:31.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.342+0000 7fb403400640 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x559fb1552980 0x559fb15269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.342+0000 7fb3f0c00640 10 client.5454 ms_handle_connect on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:31.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.342+0000 7fb403400640 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x559fb1552980 0x559fb15269d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e47d989851ab96 server_cookie=5fe731c9074fe9b8 in_seq=0 out_seq=0 2024-04-15T15:25:31.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2341752655 <== mds.0 v2:172.21.15.150:6833/3327940880 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fb146ff60 con 0x559fb1552980 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb3f0c00640 10 client.5454 handle_command_reply: tid=0 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 client.5454 shutdown 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2341752655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524860 msgr2=0x559fb1553630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524860 0x559fb1553630 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005ef0 tx=0x7fb3fc000ed0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2341752655 shutdown_connections 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x559fb1552980 0x559fb15269d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb1552440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524c40 0x559fb1520450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb3f0c00640 0 client.5454 ms_handle_reset on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2341752655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524860 0x559fb1553630 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.343+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2341752655 >> 172.21.15.150:0/2341752655 conn(0x559fb14f7d40 msgr2=0x559fb1543840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.344+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2341752655 shutdown_connections 2024-04-15T15:25:31.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.344+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2341752655 wait complete. 2024-04-15T15:25:31.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.344+0000 7fb40d5f8740 20 client.5454 trim_cache size 0 max 16384 2024-04-15T15:25:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.362+0000 7fb40d5f8740 1 Processor -- start 2024-04-15T15:25:31.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.362+0000 7fb40d5f8740 1 -- start start 2024-04-15T15:25:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.363+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524860 0x559fb14fda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.363+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb14fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.363+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb1545390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.363+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041fe700 con 0x559fb1524c40 2024-04-15T15:25:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.363+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb40413b500 con 0x559fb152b110 2024-04-15T15:25:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.363+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb40413a860 con 0x559fb1524860 2024-04-15T15:25:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb14fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb14fc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55698/0 (socket says 172.21.15.150:55698) 2024-04-15T15:25:31.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 -- 172.21.15.150:0/2347440645 learned_addr learned my addr 172.21.15.150:0/2347440645 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 -- 172.21.15.150:0/2347440645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524860 msgr2=0x559fb14fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 --2- 172.21.15.150:0/2347440645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524860 0x559fb14fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 -- 172.21.15.150:0/2347440645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 msgr2=0x559fb1545390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 --2- 172.21.15.150:0/2347440645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb1545390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 -- 172.21.15.150:0/2347440645 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb1134920 con 0x559fb1524c40 2024-04-15T15:25:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb403400640 1 --2- 172.21.15.150:0/2347440645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb14fc450 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec056670 tx=0x7fb3ec00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef75a46c6b53e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb3f3400640 1 -- 172.21.15.150:0/2347440645 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0582b0 con 0x559fb1524c40 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb3f3400640 1 -- 172.21.15.150:0/2347440645 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ec058510 con 0x559fb1524c40 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.364+0000 7fb3f3400640 1 -- 172.21.15.150:0/2347440645 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec056880 con 0x559fb1524c40 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2347440645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 msgr2=0x559fb14fc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2347440645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb14fc450 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec056670 tx=0x7fb3ec00eef0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2347440645 shutdown_connections 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2347440645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb1545390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2347440645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb14fc450 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2347440645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524860 0x559fb14fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2347440645 >> 172.21.15.150:0/2347440645 conn(0x559fb14f7d40 msgr2=0x559fb1551f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2347440645 shutdown_connections 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.365+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2347440645 wait complete. 2024-04-15T15:25:31.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.366+0000 7fb40d5f8740 1 -- start start 2024-04-15T15:25:31.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.366+0000 7fb40d5f8740 1 Processor -- start 2024-04-15T15:25:31.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524860 0x559fb1565640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb15be6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb40d5f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb15bec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559fb14209a0 con 0x559fb1524c40 2024-04-15T15:25:31.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb40413aeb0 con 0x559fb152b110 2024-04-15T15:25:31.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb40d5f8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559fb1523d90 con 0x559fb1524860 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb40d5f8740 10 client.0 _ll_register_callbacks cb 0x7fb3fc005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb403400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb15be6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb403400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb15be6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55706/0 (socket says 172.21.15.150:55706) 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.367+0000 7fb403400640 1 -- 172.21.15.150:0/2980129572 learned_addr learned my addr 172.21.15.150:0/2980129572 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb408a00640 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb15bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb403400640 1 -- 172.21.15.150:0/2980129572 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524860 msgr2=0x559fb1565640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb403400640 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524860 0x559fb1565640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb403400640 1 -- 172.21.15.150:0/2980129572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 msgr2=0x559fb15bec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb403400640 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb15bec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb403400640 1 -- 172.21.15.150:0/2980129572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb1546ae0 con 0x559fb1524c40 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb403400640 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb15be6c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec059f10 tx=0x7fb3fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb4be5407a65f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb3f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2980129572 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec08ab60 con 0x559fb1524c40 2024-04-15T15:25:31.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2980129572 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb3ec08ad00 con 0x559fb1524c40 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2980129572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3ec058510 con 0x559fb1524c40 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2980129572 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb3ec05a470 con 0x559fb1524c40 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb40d5f8740 10 client.5460 fetch_fsmap learned FSMap version 17 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.368+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2980129572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559fb1544380 con 0x559fb1524c40 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2980129572 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ec058510 con 0x559fb1524c40 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2980129572 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fb3ec0629f0 con 0x559fb1524c40 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb40d5f8740 10 client.5460 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb40d5f8740 10 client.5460 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb40d5f8740 20 client.5460 populate_metadata read hostname 'smithi150' 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x559fb14fda30 0x559fb15c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb40d5f8740 4 client.5460 mds_command: new command op to 5112 tid=0 multi_id=0 [{"prefix": "quiesce path", "path": "/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b", "wait": true}] 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2980129572 --> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] -- command(tid 0: {"prefix": "quiesce path", "path": "/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b", "wait": true}) -- 0x559fb1469dd0 con 0x559fb14fda30 2024-04-15T15:25:31.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.369+0000 7fb403e00640 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x559fb14fda30 0x559fb15c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.370+0000 7fb403e00640 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x559fb14fda30 0x559fb15c8140 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89674a0d92ee412 server_cookie=515d72e056cd7074 in_seq=0 out_seq=0 2024-04-15T15:25:31.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.370+0000 7fb3f0c00640 10 client.5460 ms_handle_connect on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:31.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.376+0000 7fb3f0c00640 1 -- 172.21.15.150:0/2980129572 <== mds.0 v2:172.21.15.150:6833/3327940880 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2332 (crc 0 0 0) 0x559fb1469dd0 con 0x559fb14fda30 2024-04-15T15:25:31.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.376+0000 7fb3f0c00640 10 client.5460 handle_command_reply: tid=0 2024-04-15T15:25:31.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 client.5460 shutdown 2024-04-15T15:25:31.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2980129572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 msgr2=0x559fb15be6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb15be6c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec059f10 tx=0x7fb3fc012000 comp rx=0 tx=0).stop 2024-04-15T15:25:31.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2980129572 shutdown_connections 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x559fb14fda30 0x559fb15c8140 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559fb152b110 0x559fb15bec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559fb1524c40 0x559fb15be6c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 --2- 172.21.15.150:0/2980129572 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559fb1524860 0x559fb1565640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb3f0c00640 0 client.5460 ms_handle_reset on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2980129572 >> 172.21.15.150:0/2980129572 conn(0x559fb14f7d40 msgr2=0x559fb1552080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2980129572 shutdown_connections 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 1 -- 172.21.15.150:0/2980129572 wait complete. 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.377+0000 7fb40d5f8740 20 client.5460 trim_cache size 0 max 16384 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "op": { 2024-04-15T15:25:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "result": 0, 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "quiesce complete", 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "reqid": { 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "entity": { 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "mds", 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "num": 0 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "op_type": "internal_op", 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "internal_op": 5384, 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "op_name": "quiesce_path", 2024-04-15T15:25:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371268+0000", 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated" 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371268+0000", 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read" 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371268+0000", 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled" 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371268+0000", 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read" 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371268+0000", 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched" 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371371+0000", 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "acquired locks" 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371393+0000", 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "acquired locks" 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371414+0000", 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "acquired locks" 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.371435+0000", 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "acquired locks" 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.377573+0000", 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "acquired locks" 2024-04-15T15:25:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.377593+0000", 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "acquired locks" 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.377612+0000", 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "acquired locks" 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.377631+0000", 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "acquired locks" 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-04-15T15:25:31.377656+0000", 2024-04-15T15:25:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "quiesce complete" 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "locks": null 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "state": { 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 2, 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_quiesced": 1, 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_blocked": 1, 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_dropped": 0, 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "failed": [] 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-15T15:25:31.386 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:25:31.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 -- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3e006c980 msgr2=0x7fb3e006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3e006c980 0x7fb3e006ee40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005880 tx=0x7fb3fc009040 comp rx=0 tx=0).stop 2024-04-15T15:25:31.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 -- 172.21.15.150:0/1317081483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040cfa80 msgr2=0x7fb40414c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040cfa80 0x7fb40414c8f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec007ef0 tx=0x7fb3ec00e4b0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 -- 172.21.15.150:0/1317081483 shutdown_connections 2024-04-15T15:25:31.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4041460f0 0x7fb40413cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040cfa80 0x7fb40414c8f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb3e006c980 0x7fb3e006ee40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 --2- 172.21.15.150:0/1317081483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4040b0b90 0x7fb40415e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 -- 172.21.15.150:0/1317081483 >> 172.21.15.150:0/1317081483 conn(0x7fb4040b3370 msgr2=0x7fb40406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:31.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 -- 172.21.15.150:0/1317081483 shutdown_connections 2024-04-15T15:25:31.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.378+0000 7fb409400640 1 -- 172.21.15.150:0/1317081483 wait complete. 2024-04-15T15:25:31.399 DEBUG:tasks.cephfs.test_quiesce:{'op': {'result': 0, 'flag_point': 'quiesce complete', 'reqid': {'entity': {'type': 'mds', 'num': 0}, 'tid': 1}, 'op_type': 'internal_op', 'internal_op': 5384, 'op_name': 'quiesce_path', 'events': [{'time': '2024-04-15T15:25:31.371268+0000', 'event': 'initiated'}, {'time': '2024-04-15T15:25:31.371268+0000', 'event': 'header_read'}, {'time': '2024-04-15T15:25:31.371268+0000', 'event': 'throttled'}, {'time': '2024-04-15T15:25:31.371268+0000', 'event': 'all_read'}, {'time': '2024-04-15T15:25:31.371268+0000', 'event': 'dispatched'}, {'time': '2024-04-15T15:25:31.371371+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:25:31.371393+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:25:31.371414+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:25:31.371435+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:25:31.377573+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:25:31.377593+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:25:31.377612+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:25:31.377631+0000', 'event': 'acquired locks'}, {'time': '2024-04-15T15:25:31.377656+0000', 'event': 'quiesce complete'}], 'locks': None}, 'state': {'inodes': 2, 'inodes_quiesced': 1, 'inodes_blocked': 1, 'inodes_dropped': 0, 'failed': []}} 2024-04-15T15:25:31.399 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:31.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.604+0000 7f9630a00640 1 Processor -- start 2024-04-15T15:25:31.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.604+0000 7f9630a00640 1 -- start start 2024-04-15T15:25:31.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.605+0000 7f9630a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c072660 0x7f962c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.605+0000 7f9630a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f962c162380 0x7f962c0af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.605+0000 7f9630a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f962c162940 con 0x7f962c148130 2024-04-15T15:25:31.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.605+0000 7f9630a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f962c148510 con 0x7f962c162380 2024-04-15T15:25:31.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.605+0000 7f9630a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f962c15fd70 con 0x7f962c072660 2024-04-15T15:25:31.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.606+0000 7f962b200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f962c162380 0x7f962c0af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.606+0000 7f962b200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f962c162380 0x7f962c0af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55710/0 (socket says 172.21.15.150:55710) 2024-04-15T15:25:31.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.606+0000 7f962b200640 1 -- 172.21.15.150:0/68544628 learned_addr learned my addr 172.21.15.150:0/68544628 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:31.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.606+0000 7f9629e00640 1 --2- 172.21.15.150:0/68544628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c072660 0x7f962c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.606+0000 7f962b200640 1 -- 172.21.15.150:0/68544628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c072660 msgr2=0x7f962c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:31.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.606+0000 7f962b200640 1 --2- 172.21.15.150:0/68544628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c072660 0x7f962c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.606+0000 7f962b200640 1 -- 172.21.15.150:0/68544628 >> v1:172.21.15.150:6789/0 conn(0x7f962c148130 legacy=0x7f962c071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.606+0000 7f962b200640 1 -- 172.21.15.150:0/68544628 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c162c20 con 0x7f962c162380 2024-04-15T15:25:31.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.608+0000 7f962b200640 1 --2- 172.21.15.150:0/68544628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c162380 0x7f962c0af1b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9624009e50 tx=0x7f962402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934bed49ee61d806 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.608+0000 7f9629400640 1 -- 172.21.15.150:0/68544628 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962402cb00 con 0x7f962c162380 2024-04-15T15:25:31.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.608+0000 7f9629400640 1 -- 172.21.15.150:0/68544628 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962402cca0 con 0x7f962c162380 2024-04-15T15:25:31.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.608+0000 7f9629400640 1 -- 172.21.15.150:0/68544628 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624034610 con 0x7f962c162380 2024-04-15T15:25:31.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.609+0000 7f9630a00640 1 -- 172.21.15.150:0/68544628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c162380 msgr2=0x7f962c0af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.609+0000 7f9630a00640 1 --2- 172.21.15.150:0/68544628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c162380 0x7f962c0af1b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9624009e50 tx=0x7f962402bf60 comp rx=0 tx=0).stop 2024-04-15T15:25:31.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.609+0000 7f9630a00640 1 -- 172.21.15.150:0/68544628 shutdown_connections 2024-04-15T15:25:31.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.609+0000 7f9630a00640 1 --2- 172.21.15.150:0/68544628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c072660 0x7f962c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.609+0000 7f9630a00640 1 --2- 172.21.15.150:0/68544628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c162380 0x7f962c0af1b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.609+0000 7f9630a00640 1 -- 172.21.15.150:0/68544628 >> 172.21.15.150:0/68544628 conn(0x7f962c06b930 msgr2=0x7f962c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:31.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.609+0000 7f9630a00640 1 -- 172.21.15.150:0/68544628 shutdown_connections 2024-04-15T15:25:31.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.609+0000 7f9630a00640 1 -- 172.21.15.150:0/68544628 wait complete. 2024-04-15T15:25:31.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.610+0000 7f9630a00640 1 Processor -- start 2024-04-15T15:25:31.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.611+0000 7f9630a00640 1 -- start start 2024-04-15T15:25:31.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.611+0000 7f9630a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f962c072660 0x7f962c1f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9630a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c148130 0x7f962c145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9630a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c162380 0x7f962c13ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9630a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f962c0aeb90 con 0x7f962c148130 2024-04-15T15:25:31.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9630a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1631e0 con 0x7f962c072660 2024-04-15T15:25:31.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9630a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f962c0aed10 con 0x7f962c162380 2024-04-15T15:25:31.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9629e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c148130 0x7f962c145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9629e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c148130 0x7f962c145be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55722/0 (socket says 172.21.15.150:55722) 2024-04-15T15:25:31.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9629e00640 1 -- 172.21.15.150:0/2461454754 learned_addr learned my addr 172.21.15.150:0/2461454754 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:31.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9629e00640 1 -- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c162380 msgr2=0x7f962c13ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9629e00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c162380 0x7f962c13ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f962a800640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f962c072660 0x7f962c1f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9629e00640 1 -- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f962c072660 msgr2=0x7f962c1f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9629e00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f962c072660 0x7f962c1f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.612+0000 7f9629e00640 1 -- 172.21.15.150:0/2461454754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c1468f0 con 0x7f962c148130 2024-04-15T15:25:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.613+0000 7f9629e00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c148130 0x7f962c145be0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f961800d830 tx=0x7f961800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.613+0000 7f9622a00640 1 -- 172.21.15.150:0/2461454754 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961800b6c0 con 0x7f962c148130 2024-04-15T15:25:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.613+0000 7f9622a00640 1 -- 172.21.15.150:0/2461454754 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9618019070 con 0x7f962c148130 2024-04-15T15:25:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.613+0000 7f9622a00640 1 -- 172.21.15.150:0/2461454754 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961800ba80 con 0x7f962c148130 2024-04-15T15:25:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.613+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c147450 con 0x7f962c148130 2024-04-15T15:25:31.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.613+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c146bb0 con 0x7f962c148130 2024-04-15T15:25:31.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.614+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c0755c0 con 0x7f962c148130 2024-04-15T15:25:31.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.615+0000 7f9622a00640 1 -- 172.21.15.150:0/2461454754 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f961801e460 con 0x7f962c148130 2024-04-15T15:25:31.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.616+0000 7f9622a00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f960c06c980 0x7f960c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:31.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.616+0000 7f9622a00640 1 -- 172.21.15.150:0/2461454754 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f961808c360 con 0x7f962c148130 2024-04-15T15:25:31.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.617+0000 7f962a800640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f960c06c980 0x7f960c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:31.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.618+0000 7f962a800640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f960c06c980 0x7f960c06ee40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f961c000f60 tx=0x7f961c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:31.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.621+0000 7f9622a00640 1 -- 172.21.15.150:0/2461454754 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f961805a6a0 con 0x7f962c148130 2024-04-15T15:25:31.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.862+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f962c06f6d0 con 0x7f962c148130 2024-04-15T15:25:31.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.864+0000 7f9622a00640 1 -- 172.21.15.150:0/2461454754 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f9618059fa0 con 0x7f962c148130 2024-04-15T15:25:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:31.866 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:09.435623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6835/3327940880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3327940880},{"type":"v1","addr":"172.21.15.150:6835","nonce":3327940880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5112,"qdb_cluster":[5112]},"id":2}]} 2024-04-15T15:25:31.866 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 17 2024-04-15T15:25:31.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f960c06c980 msgr2=0x7f960c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f960c06c980 0x7f960c06ee40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f961c000f60 tx=0x7f961c009040 comp rx=0 tx=0).stop 2024-04-15T15:25:31.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c148130 msgr2=0x7f962c145be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:31.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c148130 0x7f962c145be0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f961800d830 tx=0x7f961800dcf0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 shutdown_connections 2024-04-15T15:25:31.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f960c06c980 0x7f960c06ee40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f962c162380 0x7f962c13ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962c148130 0x7f962c145be0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 --2- 172.21.15.150:0/2461454754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f962c072660 0x7f962c1f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:31.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 >> 172.21.15.150:0/2461454754 conn(0x7f962c06b930 msgr2=0x7f962c15f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:31.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.866+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 shutdown_connections 2024-04-15T15:25:31.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:31.867+0000 7f9630a00640 1 -- 172.21.15.150:0/2461454754 wait complete. 2024-04-15T15:25:31.888 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b 0 2024-04-15T15:25:32.124 INFO:teuthology.orchestra.run.smithi150.stdout:[{"ino":1099511627779,"rdev":0,"ctime":"2024-04-15T15:25:30.966633+0000","btime":"2024-04-15T15:25:18.384341+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":5,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-15T15:25:30.966633+0000","atime":"2024-04-15T15:25:18.384341+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-15T15:25:30.966633+0000","num_files":1,"num_subdirs":0,"change_attr":1},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-15T15:25:31.087630+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-15T15:25:31.087630+0000"},"version":22,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":2,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":1},"nref":8,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{"reqid":{"entity":{"type":"mds","num":0},"tid":2}}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":4673,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8},{"client_id":4676,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8},{"client_id":4837,"pending":"pAsLsXs","issued":"pAsLsXs","wanted":"-","last_sent":9},{"client_id":4888,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFsxcral","last_sent":30},{"client_id":5424,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/pyvenv.cfg","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":1,"projected_version":0,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"lock":0,"authpin":0},"nref":0,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth"]},{"path":"volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/file","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629282,"remote_type":"","version":6,"projected_version":6,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-15T15:25:32.125 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-15T15:25:31.087630+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-15T15:25:18.384341+0000", "auth_pins": 1, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-15T15:25:18.384341+0000", "change_attr": 3, "client_caps": [ { "client_id": 4673, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" }, { "client_id": 4676, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" }, { "client_id": 4837, "issued": "pAsLsXs", "last_sent": 9, "pending": "pAsLsXs", "wanted": "-" }, { "client_id": 4888, "issued": "pAsLsXsFs", "last_sent": 30, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFsxcral" }, { "client_id": 5424, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" } ], "client_ranges": [], "ctime": "2024-04-15T15:25:30.966633+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 0, "path": "volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/pyvenv.cfg", "path_ino": 1, "pins": { "authpin": 0, "lock": 0 }, "projected_version": 0, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth" ], "version": 1, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629282, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/file", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 6, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 6, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 1, "mtime": "2024-04-15T15:25:30.966633+0000", "num_files": 1, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1099511627779, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 5, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2024-04-15T15:25:30.966633+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 8, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 2, "request": 1, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "lock", "type": "iquiesce", "xlock_by": { "reqid": { "entity": { "num": 0, "type": "mds" }, "tid": 2 } } }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-15T15:25:31.087630+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 22, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-04-15T15:25:32.126 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 ops --flags=locks --path=/tmp/mds.0-ops 2024-04-15T15:25:32.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.346+0000 7f92dae00640 1 Processor -- start 2024-04-15T15:25:32.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.346+0000 7f92dae00640 1 -- start start 2024-04-15T15:25:32.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.347+0000 7f92dae00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f92d409ab90 0x7f92d40b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.347+0000 7f92dae00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d40b6a20 con 0x7f92d40d0b90 2024-04-15T15:25:32.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.347+0000 7f92dae00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d40b6fa0 con 0x7f92d40b3a20 2024-04-15T15:25:32.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.347+0000 7f92dae00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f92d40cf6f0 con 0x7f92d409ab90 2024-04-15T15:25:32.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.348+0000 7f92d8e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f92d40d0b90 0x7f92d4166460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:43106/0 (socket says 172.21.15.150:43106) 2024-04-15T15:25:32.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.348+0000 7f92d8e00640 1 -- 172.21.15.150:0/805849408 learned_addr learned my addr 172.21.15.150:0/805849408 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.348+0000 7f92d3e00640 1 --2- 172.21.15.150:0/805849408 >> v2:172.21.15.187:3300/0 conn(0x7f92d409ab90 0x7f92d40b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.348+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559276373 0 0) 0x7f92d40b6a20 con 0x7f92d40d0b90 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.348+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b8003520 con 0x7f92d40d0b90 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2575879127 0 0) 0x7f92b8003520 con 0x7f92d40d0b90 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 >> v2:172.21.15.187:3300/0 conn(0x7f92d409ab90 msgr2=0x7f92d40b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 --2- 172.21.15.150:0/805849408 >> v2:172.21.15.187:3300/0 conn(0x7f92d409ab90 0x7f92d40b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 >> v1:172.21.15.187:6789/0 conn(0x7f92d40b3a20 legacy=0x7f92d409af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d40b6cc0 con 0x7f92d40d0b90 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f92cc001670 con 0x7f92d40d0b90 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 >> v1:172.21.15.150:6789/0 conn(0x7f92d40d0b90 legacy=0x7f92d4166460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92b80039b0 0x7f92b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b8004800 0x7f92b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b80051c0 0x7f92b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92d40cf6f0 con 0x7f92b8004800 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f92b800c060 con 0x7f92b80039b0 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f92b800c260 con 0x7f92b80051c0 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d3400640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b8004800 0x7f92b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.349+0000 7f92d3e00640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92b80039b0 0x7f92b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.350+0000 7f92d8e00640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b80051c0 0x7f92b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.350+0000 7f92d3400640 1 -- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b80051c0 msgr2=0x7f92b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.350+0000 7f92d3400640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b80051c0 0x7f92b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.350+0000 7f92d3400640 1 -- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92b80039b0 msgr2=0x7f92b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.350+0000 7f92d3400640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92b80039b0 0x7f92b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.350+0000 7f92d3400640 1 -- 172.21.15.150:0/805849408 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b8003520 con 0x7f92b8004800 2024-04-15T15:25:32.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.350+0000 7f92d8e00640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b80051c0 0x7f92b800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:25:32.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.350+0000 7f92d8e00640 1 -- 172.21.15.150:0/805849408 reap_dead start 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.351+0000 7f92d3400640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b8004800 0x7f92b8004c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f92bc001e50 tx=0x7f92b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88fa738dc1c5293 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.351+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc001a70 con 0x7f92b8004800 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.351+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92bc003b70 con 0x7f92b8004800 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.351+0000 7f92d2a00640 1 -- 172.21.15.150:0/805849408 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0050d0 con 0x7f92b8004800 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.352+0000 7f92dae00640 1 -- 172.21.15.150:0/805849408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b8004800 msgr2=0x7f92b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.352+0000 7f92dae00640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b8004800 0x7f92b8004c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f92bc001e50 tx=0x7f92b8005830 comp rx=0 tx=0).stop 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.352+0000 7f92dae00640 1 -- 172.21.15.150:0/805849408 shutdown_connections 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.352+0000 7f92dae00640 1 --2- 172.21.15.150:0/805849408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92b8004800 0x7f92b8004c80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.352+0000 7f92dae00640 1 -- 172.21.15.150:0/805849408 >> 172.21.15.150:0/805849408 conn(0x7f92d40b7e40 msgr2=0x7f92d40b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:32.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.352+0000 7f92dae00640 1 -- 172.21.15.150:0/805849408 shutdown_connections 2024-04-15T15:25:32.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.352+0000 7f92dae00640 1 -- 172.21.15.150:0/805849408 wait complete. 2024-04-15T15:25:32.356 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.354+0000 7f92dae00640 1 Processor -- start 2024-04-15T15:25:32.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.354+0000 7f92dae00640 1 -- start start 2024-04-15T15:25:32.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92dae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b8004800 0x7f92d4142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92dae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92d4155500 0x7f92d41460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92dae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d40b4390 0x7f92d40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92dae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92d40b6290 con 0x7f92d40b4390 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92dae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f92d40b6110 con 0x7f92d4155500 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92dae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f92d40b6410 con 0x7f92b8004800 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92d8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d40b4390 0x7f92d40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92d8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d40b4390 0x7f92d40b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55736/0 (socket says 172.21.15.150:55736) 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92d8e00640 1 -- 172.21.15.150:0/512039465 learned_addr learned my addr 172.21.15.150:0/512039465 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92d3400640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92d4155500 0x7f92d41460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.355+0000 7f92d3e00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b8004800 0x7f92d4142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92d8e00640 1 -- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b8004800 msgr2=0x7f92d4142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92d8e00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b8004800 0x7f92d4142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92d8e00640 1 -- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92d4155500 msgr2=0x7f92d41460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92d8e00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92d4155500 0x7f92d41460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92d8e00640 1 -- 172.21.15.150:0/512039465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d4155960 con 0x7f92d40b4390 2024-04-15T15:25:32.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92d3e00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b8004800 0x7f92d4142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:32.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92d8e00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d40b4390 0x7f92d40b4810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f92d40b7bd0 tx=0x7f92cc00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:32.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92cbe00640 1 -- 172.21.15.150:0/512039465 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc019070 con 0x7f92d40b4390 2024-04-15T15:25:32.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92cbe00640 1 -- 172.21.15.150:0/512039465 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92cc0115b0 con 0x7f92d40b4390 2024-04-15T15:25:32.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92cbe00640 1 -- 172.21.15.150:0/512039465 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0119a0 con 0x7f92d40b4390 2024-04-15T15:25:32.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92dae00640 1 -- 172.21.15.150:0/512039465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d4155bf0 con 0x7f92d40b4390 2024-04-15T15:25:32.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.356+0000 7f92dae00640 1 -- 172.21.15.150:0/512039465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d41435e0 con 0x7f92d40b4390 2024-04-15T15:25:32.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.359+0000 7f92cbe00640 1 -- 172.21.15.150:0/512039465 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f92cc01c070 con 0x7f92d40b4390 2024-04-15T15:25:32.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.359+0000 7f92cbe00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f92b006c980 0x7f92b006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.360+0000 7f92cbe00640 1 -- 172.21.15.150:0/512039465 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f92cc08b1c0 con 0x7f92d40b4390 2024-04-15T15:25:32.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.360+0000 7f92ddbda740 1 Processor -- start 2024-04-15T15:25:32.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.360+0000 7f92d3e00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f92b006c980 0x7f92b006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.360+0000 7f92ddbda740 1 -- start start 2024-04-15T15:25:32.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 0x55d5020dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92ddbda740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d5020167b0 con 0x55d5020dc010 2024-04-15T15:25:32.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92ddbda740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d501fbd2d0 con 0x55d5020d9430 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92ddbda740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d502012770 con 0x55d5020b1140 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92d3e00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f92b006c980 0x7f92b006ee40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009b30 tx=0x7f92c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92d3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92d3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55742/0 (socket says 172.21.15.150:55742) 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.361+0000 7f92d3e00640 1 -- 172.21.15.150:0/12154731 learned_addr learned my addr 172.21.15.150:0/12154731 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92d3400640 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 0x55d5020dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92d8e00640 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92d3e00640 1 -- 172.21.15.150:0/12154731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 msgr2=0x55d5020d88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92d3e00640 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92d3e00640 1 -- 172.21.15.150:0/12154731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 msgr2=0x55d5020dc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92d3e00640 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 0x55d5020dc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92d3e00640 1 -- 172.21.15.150:0/12154731 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d501d29420 con 0x55d5020dc010 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92d3e00640 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020b2ea0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f92c4013e70 tx=0x7f92c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04b8a667daae518 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92c8c00640 1 -- 172.21.15.150:0/12154731 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4029070 con 0x55d5020dc010 2024-04-15T15:25:32.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92c8c00640 1 -- 172.21.15.150:0/12154731 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c401f050 con 0x55d5020dc010 2024-04-15T15:25:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.362+0000 7f92c8c00640 1 -- 172.21.15.150:0/12154731 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40044c0 con 0x55d5020dc010 2024-04-15T15:25:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 -- 172.21.15.150:0/12154731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 msgr2=0x55d5020b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020b2ea0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f92c4013e70 tx=0x7f92c4016000 comp rx=0 tx=0).stop 2024-04-15T15:25:32.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 -- 172.21.15.150:0/12154731 shutdown_connections 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 0x55d5020dc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 --2- 172.21.15.150:0/12154731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020b2ea0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 -- 172.21.15.150:0/12154731 >> 172.21.15.150:0/12154731 conn(0x55d502046f80 msgr2=0x55d5020f6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 -- 172.21.15.150:0/12154731 shutdown_connections 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.363+0000 7f92ddbda740 1 -- 172.21.15.150:0/12154731 wait complete. 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.364+0000 7f92ddbda740 1 -- start start 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.364+0000 7f92ddbda740 1 Processor -- start 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020d9430 0x55d5020d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020dc010 0x55d5020d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92ddbda740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d5020ac440 con 0x55d5020d9430 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92ddbda740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d501fd06a0 con 0x55d5020dc010 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92ddbda740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d5020f75d0 con 0x55d5020b1140 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92ddbda740 10 client.0 _ll_register_callbacks cb 0x55d501fd4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92d3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020d9430 0x55d5020d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92d3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020d9430 0x55d5020d65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55754/0 (socket says 172.21.15.150:55754) 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.365+0000 7f92d3400640 1 -- 172.21.15.150:0/1884193011 learned_addr learned my addr 172.21.15.150:0/1884193011 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d3e00640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d8e00640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020dc010 0x55d5020d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d3400640 1 -- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 msgr2=0x55d5020d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d3400640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d3400640 1 -- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020dc010 msgr2=0x55d5020d6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d3400640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020dc010 0x55d5020d6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d3400640 1 -- 172.21.15.150:0/1884193011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d501df5830 con 0x55d5020d9430 2024-04-15T15:25:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d3e00640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d8e00640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020dc010 0x55d5020d6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92d3400640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020d9430 0x55d5020d65f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005040 tx=0x7f92bc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c899dfd6ba1252c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92c1600640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92c1600640 1 -- 172.21.15.150:0/1884193011 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc006d50 con 0x55d5020d9430 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92c1600640 1 -- 172.21.15.150:0/1884193011 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f92bc0050d0 con 0x55d5020d9430 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.366+0000 7f92ddbda740 1 -- 172.21.15.150:0/1884193011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c401f050 con 0x55d5020d9430 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92c1600640 1 -- 172.21.15.150:0/1884193011 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f92bc04d050 con 0x55d5020d9430 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92ddbda740 10 client.5481 fetch_fsmap learned FSMap version 17 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92ddbda740 1 -- 172.21.15.150:0/1884193011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d5020d28d0 con 0x55d5020d9430 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92c1600640 1 -- 172.21.15.150:0/1884193011 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c401f050 con 0x55d5020d9430 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92c1600640 1 -- 172.21.15.150:0/1884193011 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f92bc050020 con 0x55d5020d9430 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92ddbda740 10 client.5481 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92ddbda740 10 client.5481 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-15T15:25:32.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92ddbda740 20 client.5481 populate_metadata read hostname 'smithi150' 2024-04-15T15:25:32.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55d5020dd8d0 0x55d5020ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92ddbda740 4 client.5481 mds_command: new command op to 5112 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:25:32.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92ddbda740 1 -- 172.21.15.150:0/1884193011 --> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5020241d0 con 0x55d5020dd8d0 2024-04-15T15:25:32.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.367+0000 7f92d3e00640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55d5020dd8d0 0x55d5020ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.368+0000 7f92d3e00640 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55d5020dd8d0 0x55d5020ddcd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbec3818d30063e4 server_cookie=ecae7a21b2f2ba7 in_seq=0 out_seq=0 2024-04-15T15:25:32.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.368+0000 7f92c1600640 10 client.5481 ms_handle_connect on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92c1600640 1 -- 172.21.15.150:0/1884193011 <== mds.0 v2:172.21.15.150:6833/3327940880 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5020241d0 con 0x55d5020dd8d0 2024-04-15T15:25:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92c1600640 10 client.5481 handle_command_reply: tid=0 2024-04-15T15:25:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 client.5481 shutdown 2024-04-15T15:25:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 -- 172.21.15.150:0/1884193011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020d9430 msgr2=0x55d5020d65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020d9430 0x55d5020d65f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005040 tx=0x7f92bc02e000 comp rx=0 tx=0).stop 2024-04-15T15:25:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 -- 172.21.15.150:0/1884193011 shutdown_connections 2024-04-15T15:25:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55d5020dd8d0 0x55d5020ddcd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020dc010 0x55d5020d6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020d9430 0x55d5020d65f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1884193011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92c1600640 0 client.5481 ms_handle_reset on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 -- 172.21.15.150:0/1884193011 >> 172.21.15.150:0/1884193011 conn(0x55d502046f80 msgr2=0x55d5020b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 -- 172.21.15.150:0/1884193011 shutdown_connections 2024-04-15T15:25:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 1 -- 172.21.15.150:0/1884193011 wait complete. 2024-04-15T15:25:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.369+0000 7f92ddbda740 20 client.5481 trim_cache size 0 max 16384 2024-04-15T15:25:32.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.387+0000 7f92ddbda740 1 Processor -- start 2024-04-15T15:25:32.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.387+0000 7f92ddbda740 1 -- start start 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.388+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020b1140 0x55d50211b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.388+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020d9430 0x55d5020d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.388+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020d8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.388+0000 7f92ddbda740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41ff520 con 0x55d5020dc010 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.388+0000 7f92ddbda740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41fe530 con 0x55d5020b1140 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.388+0000 7f92ddbda740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f92d4144e10 con 0x55d5020d9430 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020d8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020d8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55758/0 (socket says 172.21.15.150:55758) 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 -- 172.21.15.150:0/1761376734 learned_addr learned my addr 172.21.15.150:0/1761376734 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:32.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 -- 172.21.15.150:0/1761376734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020d9430 msgr2=0x55d5020d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 --2- 172.21.15.150:0/1761376734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020d9430 0x55d5020d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 -- 172.21.15.150:0/1761376734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020b1140 msgr2=0x55d50211b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 --2- 172.21.15.150:0/1761376734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020b1140 0x55d50211b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 -- 172.21.15.150:0/1761376734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5020ccc10 con 0x55d5020dc010 2024-04-15T15:25:32.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.389+0000 7f92d8e00640 1 --2- 172.21.15.150:0/1761376734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020d8a30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0895b0 tx=0x7f92cc058790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3285bfdd0c10aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92c3e00640 1 -- 172.21.15.150:0/1761376734 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc087070 con 0x55d5020dc010 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92c3e00640 1 -- 172.21.15.150:0/1761376734 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92cc05d120 con 0x55d5020dc010 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92c3e00640 1 -- 172.21.15.150:0/1761376734 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0619e0 con 0x55d5020dc010 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 -- 172.21.15.150:0/1761376734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 msgr2=0x55d5020d8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1761376734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020d8a30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0895b0 tx=0x7f92cc058790 comp rx=0 tx=0).stop 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 -- 172.21.15.150:0/1761376734 shutdown_connections 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1761376734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d5020d8a30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1761376734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020d9430 0x55d5020d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 --2- 172.21.15.150:0/1761376734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020b1140 0x55d50211b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 -- 172.21.15.150:0/1761376734 >> 172.21.15.150:0/1761376734 conn(0x55d502046f80 msgr2=0x55d5020d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 -- 172.21.15.150:0/1761376734 shutdown_connections 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.390+0000 7f92ddbda740 1 -- 172.21.15.150:0/1761376734 wait complete. 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.391+0000 7f92ddbda740 1 -- start start 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.391+0000 7f92ddbda740 1 Processor -- start 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.392+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.392+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 0x55d502105bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.392+0000 7f92ddbda740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d502106100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.392+0000 7f92ddbda740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41fe030 con 0x55d5020dc010 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.392+0000 7f92ddbda740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d5020cb930 con 0x55d5020d9430 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.392+0000 7f92ddbda740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d5020cbad0 con 0x55d5020b1140 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.392+0000 7f92ddbda740 10 client.0 _ll_register_callbacks cb 0x7f92b80036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:25:32.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.392+0000 7f92d8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d502106100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d502106100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55762/0 (socket says 172.21.15.150:55762) 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d8e00640 1 -- 172.21.15.150:0/621152771 learned_addr learned my addr 172.21.15.150:0/621152771 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d3e00640 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d8e00640 1 -- 172.21.15.150:0/621152771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 msgr2=0x55d5020cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d8e00640 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d3400640 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 0x55d502105bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d8e00640 1 -- 172.21.15.150:0/621152771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 msgr2=0x55d502105bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d8e00640 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 0x55d502105bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d8e00640 1 -- 172.21.15.150:0/621152771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d502176250 con 0x55d5020dc010 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92d8e00640 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d502106100 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f92cc08b170 tx=0x7f92cc08b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd251c852a6a658 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92c1600640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92c1600640 1 -- 172.21.15.150:0/621152771 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc087050 con 0x55d5020dc010 2024-04-15T15:25:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92c1600640 1 -- 172.21.15.150:0/621152771 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f92cc079dd0 con 0x55d5020dc010 2024-04-15T15:25:32.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.393+0000 7f92ddbda740 1 -- 172.21.15.150:0/621152771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92cc05d120 con 0x55d5020dc010 2024-04-15T15:25:32.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92c1600640 1 -- 172.21.15.150:0/621152771 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f92cc0553e0 con 0x55d5020dc010 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92ddbda740 10 client.5487 fetch_fsmap learned FSMap version 17 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92ddbda740 1 -- 172.21.15.150:0/621152771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d5020b3400 con 0x55d5020dc010 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92c1600640 1 -- 172.21.15.150:0/621152771 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92cc05d120 con 0x55d5020dc010 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92c1600640 1 -- 172.21.15.150:0/621152771 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f92cc0782b0 con 0x55d5020dc010 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92ddbda740 10 client.5487 fetch_fsmap finished waiting for FSMap version 17 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92ddbda740 10 client.5487 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92ddbda740 20 client.5487 populate_metadata read hostname 'smithi150' 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92ddbda740 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55d5020afa90 0x55d502179630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92ddbda740 4 client.5487 mds_command: new command op to 5112 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.0-ops"}] 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92ddbda740 1 -- 172.21.15.150:0/621152771 --> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.0-ops"}) -- 0x55d50201e040 con 0x55d5020afa90 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.394+0000 7f92d3e00640 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55d5020afa90 0x55d502179630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.395+0000 7f92d3e00640 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55d5020afa90 0x55d502179630 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887782989a62211b server_cookie=edf66e81e3cc3dce in_seq=0 out_seq=0 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.395+0000 7f92c1600640 10 client.5487 ms_handle_connect on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92c1600640 1 -- 172.21.15.150:0/621152771 <== mds.0 v2:172.21.15.150:6833/3327940880 1 ==== command_reply(tid 0: 0 ) ==== 8+0+33 (crc 0 0 0) 0x55d50201e040 con 0x55d5020afa90 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92c1600640 10 client.5487 handle_command_reply: tid=0 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 client.5487 shutdown 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 -- 172.21.15.150:0/621152771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 msgr2=0x55d502106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d502106100 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f92cc08b170 tx=0x7f92cc08b040 comp rx=0 tx=0).stop 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 -- 172.21.15.150:0/621152771 shutdown_connections 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.150:6833/3327940880,v1:172.21.15.150:6835/3327940880] conn(0x55d5020afa90 0x55d502179630 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d5020dc010 0x55d502106100 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5020d9430 0x55d502105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 --2- 172.21.15.150:0/621152771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d5020b1140 0x55d5020cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92c1600640 0 client.5487 ms_handle_reset on v2:172.21.15.150:6833/3327940880 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 -- 172.21.15.150:0/621152771 >> 172.21.15.150:0/621152771 conn(0x55d502046f80 msgr2=0x55d5020d8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 -- 172.21.15.150:0/621152771 shutdown_connections 2024-04-15T15:25:32.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 1 -- 172.21.15.150:0/621152771 wait complete. 2024-04-15T15:25:32.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.396+0000 7f92ddbda740 20 client.5487 trim_cache size 0 max 16384 2024-04-15T15:25:32.402 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:25:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/tmp/mds.0-ops" 2024-04-15T15:25:32.402 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:25:32.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 -- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f92b006c980 msgr2=0x7f92b006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f92b006c980 0x7f92b006ee40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009b30 tx=0x7f92c4008040 comp rx=0 tx=0).stop 2024-04-15T15:25:32.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 -- 172.21.15.150:0/512039465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d40b4390 msgr2=0x7f92d40b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:32.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d40b4390 0x7f92d40b4810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f92d40b7bd0 tx=0x7f92cc00e960 comp rx=0 tx=0).stop 2024-04-15T15:25:32.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 -- 172.21.15.150:0/512039465 shutdown_connections 2024-04-15T15:25:32.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d40b4390 0x7f92d40b4810 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92d4155500 0x7f92d41460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f92b006c980 0x7f92b006ee40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 --2- 172.21.15.150:0/512039465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92b8004800 0x7f92d4142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:32.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 -- 172.21.15.150:0/512039465 >> 172.21.15.150:0/512039465 conn(0x7f92d40b7e40 msgr2=0x7f92d40d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:32.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 -- 172.21.15.150:0/512039465 shutdown_connections 2024-04-15T15:25:32.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:32.397+0000 7f92dae00640 1 -- 172.21.15.150:0/512039465 wait complete. 2024-04-15T15:25:32.416 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-04-15T15:25:32.458 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.maYf8mQJMe 2024-04-15T15:25:32.458 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /tmp/mds.0-ops /tmp/tmp.maYf8mQJMe 2024-04-15T15:25:32.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.maYf8mQJMe 2024-04-15T15:25:32.756 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.maYf8mQJMe is 8KB 2024-04-15T15:25:32.817 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.maYf8mQJMe 2024-04-15T15:25:32.868 DEBUG:tasks.cephfs.filesystem:read 7851B of ops 2024-04-15T15:25:32.868 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b --path=/tmp/mds.0-cache 2024-04-15T15:25:33.096 INFO:teuthology.orchestra.run.smithi150.stdout:{"path":"/tmp/mds.0-cache"} 2024-04-15T15:25:33.096 DEBUG:tasks.cephfs.filesystem:_json_asok output { "path": "/tmp/mds.0-cache" } 2024-04-15T15:25:33.097 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-04-15T15:25:33.147 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.E0Sd5JlT5U 2024-04-15T15:25:33.147 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /tmp/mds.0-cache /tmp/tmp.E0Sd5JlT5U 2024-04-15T15:25:33.240 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.E0Sd5JlT5U 2024-04-15T15:25:33.395 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.E0Sd5JlT5U is 7KB 2024-04-15T15:25:33.457 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.E0Sd5JlT5U 2024-04-15T15:25:33.509 DEBUG:tasks.cephfs.filesystem:read 7619B of cache 2024-04-15T15:25:33.509 INFO:tasks.cephfs.test_quiesce:qp = 1; qi = 1; qib = 1; qina = 0 2024-04-15T15:25:33.510 ERROR:tasks.cephfs.test_quiesce:inode: { "ino": 1099511627779, "rdev": 0, "ctime": "2024-04-15T15:25:30.966633+0000", "btime": "2024-04-15T15:25:18.384341+0000", "mode": 17407, "uid": 0, "gid": 0, "nlink": 1, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:25:30.966633+0000", "atime": "2024-04-15T15:25:18.384341+0000", "time_warp_seq": 0, "change_attr": 3, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": false, "client_ranges": [], "dirstat": { "version": 0, "mtime": "2024-04-15T15:25:30.966633+0000", "num_files": 1, "num_subdirs": 0, "change_attr": 1 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:25:31.087630+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:25:31.087630+0000" }, "version": 22, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 2, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "authlock": {}, "linklock": {}, "dirfragtreelock": {}, "filelock": {}, "xattrlock": {}, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 4673, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 }, { "client_id": 4676, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 }, { "client_id": 4837, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 9 }, { "client_id": 4888, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFsxcral", "last_sent": 30 }, { "client_id": 5424, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 } ], "loner": -1, "want_loner": -1, "mds_caps_wanted": [], "dirfrags": [ { "path": "/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b", "dirfrag": "0x10000000003", "snapid_first": 2, "projected_version": "7", "version": "7", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "dentries": [ { "path": "volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/pyvenv.cfg", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": false, "is_remote": false, "is_null": true, "is_new": false, "inode": 0, "remote_type": "", "version": 1, "projected_version": 0, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "lock": 0, "authpin": 0 }, "nref": 0, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth" ] }, { "path": "volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/file", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": true, "is_remote": false, "is_null": false, "is_new": false, "inode": 1099511629282, "remote_type": "", "version": 6, "projected_version": 6, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "request": 0, "lock": 0, "inodepin": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty" ] } ] } ], "is_subvolume": false } 2024-04-15T15:25:33.510 ERROR:tasks.cephfs.test_quiesce:op: { "description": "internal op quiesce_inode:mds.0:2 fp=#0x10000000003", "initiated_at": "2024-04-15T15:25:31.371458+0000", "age": 1.025362939, "duration": 1.025501783, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce complete", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:25:31.371458+0000", "event": "initiated" }, { "time": "2024-04-15T15:25:31.371458+0000", "event": "header_read" }, { "time": "2024-04-15T15:25:31.371458+0000", "event": "throttled" }, { "time": "2024-04-15T15:25:31.371458+0000", "event": "all_read" }, { "time": "2024-04-15T15:25:31.371458+0000", "event": "dispatched" }, { "time": "2024-04-15T15:25:31.371816+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:25:31.372845+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:25:31.377537+0000", "event": "quiesce complete" } ], "locks": [ { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/ auth v22 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:25:30.966633+0000 1=1+0) n(v0 rc2024-04-15T15:25:31.087630+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={4673=pAsLsXsFs/pAsLsXsFs@8,4676=pAsLsXsFs/pAsLsXsFs@8,4837=pAsLsXs/-@9,4888=pAsLsXsFs/pAsLsXsFsxcral@30,5424=pAsLsXsFs/pAsLsXsFs@8} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x55d87a1a0b00]", "lock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "flags": 4, "wrlock_target": -1 }, { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/ auth v22 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:25:30.966633+0000 1=1+0) n(v0 rc2024-04-15T15:25:31.087630+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={4673=pAsLsXsFs/pAsLsXsFs@8,4676=pAsLsXsFs/pAsLsXsFs@8,4837=pAsLsXs/-@9,4888=pAsLsXsFs/pAsLsXsFsxcral@30,5424=pAsLsXsFs/pAsLsXsFs@8} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x55d87a1a0b00]", "lock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "flags": 2, "wrlock_target": -1 } ] } } 2024-04-15T15:25:33.510 ERROR:tasks.cephfs.test_quiesce:lock_type: iversion 2024-04-15T15:25:33.511 ERROR:tasks.cephfs.test_quiesce:cache: [ { "ino": 1099511629282, "rdev": 0, "ctime": "2024-04-15T15:25:31.087630+0000", "btime": "2024-04-15T15:25:30.966633+0000", "mode": 33188, "uid": 1000, "gid": 1301, "nlink": 1, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:25:30.980632+0000", "atime": "2024-04-15T15:25:30.980632+0000", "time_warp_seq": 2, "change_attr": 1, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": true, "client_ranges": [ { "client": 4888, "byte range": { "first": 0, "last": 4194304 }, "follows": 1 } ], "dirstat": { "version": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "change_attr": 0 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 0, "rsnaps": 0, "rctime": "2024-04-15T15:25:31.087630+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 0, "rsnaps": 0, "rctime": "2024-04-15T15:25:31.087630+0000" }, "version": 6, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 4, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 1, "lock": 0, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 5, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "authlock": { "gather_set": [], "state": "excl", "type": "iauth", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "linklock": {}, "dirfragtreelock": {}, "filelock": { "gather_set": [], "state": "excl", "type": "ifile", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "xattrlock": { "gather_set": [], "state": "excl", "type": "ixattr", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 4888, "pending": "pAsxLsXsxFsxcrwb", "issued": "pAsxLsXsxFsxcrwb", "wanted": "pAsxXsxFxwb", "last_sent": 4 } ], "loner": 4888, "want_loner": 4888, "mds_caps_wanted": [], "dirfrags": [], "is_subvolume": false }, { "ino": 1099511627779, "rdev": 0, "ctime": "2024-04-15T15:25:30.966633+0000", "btime": "2024-04-15T15:25:18.384341+0000", "mode": 17407, "uid": 0, "gid": 0, "nlink": 1, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "layout": { "stripe_unit": 4194304, "stripe_count": 1, "object_size": 4194304, "pool_id": 5, "pool_ns": "" }, "old_pools": [], "size": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "truncate_from": 0, "truncate_pending": 0, "mtime": "2024-04-15T15:25:30.966633+0000", "atime": "2024-04-15T15:25:18.384341+0000", "time_warp_seq": 0, "change_attr": 3, "export_pin": -1, "export_ephemeral_random_pin": 0, "export_ephemeral_distributed_pin": false, "quiesce_block": false, "client_ranges": [], "dirstat": { "version": 0, "mtime": "2024-04-15T15:25:30.966633+0000", "num_files": 1, "num_subdirs": 0, "change_attr": 1 }, "rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:25:31.087630+0000" }, "accounted_rstat": { "version": 0, "rbytes": 0, "rfiles": 1, "rsubdirs": 1, "rsnaps": 0, "rctime": "2024-04-15T15:25:31.087630+0000" }, "version": 22, "file_data_version": 0, "xattr_version": 1, "backtrace_version": 2, "stray_prior_path": "", "max_size_ever": 0, "quota": { "max_bytes": 0, "max_files": 0 }, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "symlink": "", "xattrs": [], "dirfragtree": { "splits": [] }, "old_inodes": [], "oldest_snap": 18446744073709551614, "damage_flags": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8, "versionlock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "authlock": {}, "linklock": {}, "dirfragtreelock": {}, "filelock": {}, "xattrlock": {}, "snaplock": {}, "nestlock": { "gather_set": [], "state": "lock", "type": "inest", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "flocklock": {}, "policylock": {}, "quiescelock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "client_caps": [ { "client_id": 4673, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 }, { "client_id": 4676, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 }, { "client_id": 4837, "pending": "pAsLsXs", "issued": "pAsLsXs", "wanted": "-", "last_sent": 9 }, { "client_id": 4888, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFsxcral", "last_sent": 30 }, { "client_id": 5424, "pending": "pAsLsXsFs", "issued": "pAsLsXsFs", "wanted": "pAsLsXsFs", "last_sent": 8 } ], "loner": -1, "want_loner": -1, "mds_caps_wanted": [], "dirfrags": [ { "path": "/volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b", "dirfrag": "0x10000000003", "snapid_first": 2, "projected_version": "7", "version": "7", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "dentries": [ { "path": "volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/pyvenv.cfg", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": false, "is_remote": false, "is_null": true, "is_new": false, "inode": 0, "remote_type": "", "version": 1, "projected_version": 0, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "lock": 0, "authpin": 0 }, "nref": 0, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth" ] }, { "path": "volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/file", "path_ino": 1, "snap_first": 2, "snap_last": 18446744073709551614, "is_primary": true, "is_remote": false, "is_null": false, "is_new": false, "inode": 1099511629282, "remote_type": "", "version": 6, "projected_version": 6, "auth_pins": 0, "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "is_frozen": false, "is_freezing": false, "pins": { "request": 0, "lock": 0, "inodepin": 1, "dirty": 1, "authpin": 0 }, "nref": 2, "lock": {}, "versionlock": { "gather_set": [], "state": "lock", "type": "dversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "xlock_by": {} }, "states": [ "auth", "dirty" ] } ] } ], "is_subvolume": false } ] 2024-04-15T15:25:33.512 ERROR:tasks.cephfs.test_quiesce:ops: { "1099511627779": { "description": "internal op quiesce_inode:mds.0:2 fp=#0x10000000003", "initiated_at": "2024-04-15T15:25:31.371458+0000", "age": 1.025362939, "duration": 1.025501783, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce complete", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:25:31.371458+0000", "event": "initiated" }, { "time": "2024-04-15T15:25:31.371458+0000", "event": "header_read" }, { "time": "2024-04-15T15:25:31.371458+0000", "event": "throttled" }, { "time": "2024-04-15T15:25:31.371458+0000", "event": "all_read" }, { "time": "2024-04-15T15:25:31.371458+0000", "event": "dispatched" }, { "time": "2024-04-15T15:25:31.371816+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:25:31.372845+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:25:31.377537+0000", "event": "quiesce complete" } ], "locks": [ { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/ auth v22 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:25:30.966633+0000 1=1+0) n(v0 rc2024-04-15T15:25:31.087630+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={4673=pAsLsXsFs/pAsLsXsFs@8,4676=pAsLsXsFs/pAsLsXsFs@8,4837=pAsLsXs/-@9,4888=pAsLsXsFs/pAsLsXsFsxcral@30,5424=pAsLsXsFs/pAsLsXsFs@8} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x55d87a1a0b00]", "lock": { "gather_set": [], "state": "lock", "type": "iquiesce", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "xlock_by": { "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 2 } } }, "flags": 4, "wrlock_target": -1 }, { "object": { "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "request": 1, "lock": 2, "dirfrag": 1, "caps": 1, "dirtyparent": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 8 }, "object_string": "[inode 0x10000000003 [...2,head] /volumes/_nogroup/subvol_quiesce/ab3b3652-c2e3-4edb-92b0-cda322ddc16b/ auth v22 ap=1 DIRTYPARENT f(v0 m2024-04-15T15:25:30.966633+0000 1=1+0) n(v0 rc2024-04-15T15:25:31.087630+0000 2=1+1) (inest lock) (iversion lock w=1) (iquiesce lock x=1 by request(mds.0:2 nref=3)) caps={4673=pAsLsXsFs/pAsLsXsFs@8,4676=pAsLsXsFs/pAsLsXsFs@8,4837=pAsLsXs/-@9,4888=pAsLsXsFs/pAsLsXsFsxcral@30,5424=pAsLsXsFs/pAsLsXsFs@8} | request=1 lock=2 dirfrag=1 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 0x55d87a1a0b00]", "lock": { "gather_set": [], "state": "lock", "type": "iversion", "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "xlock_by": {} }, "flags": 2, "wrlock_target": -1 } ] } }, "1099511629282": { "description": "internal op quiesce_inode:mds.0:3 fp=#0x100000005e2", "initiated_at": "2024-04-15T15:25:31.373265+0000", "age": 1.023556364, "duration": 1.023807356, "continuous": true, "type_data": { "result": -2147483648, "flag_point": "quiesce blocked", "reqid": { "entity": { "type": "mds", "num": 0 }, "tid": 3 }, "op_type": "internal_op", "internal_op": 5385, "op_name": "quiesce_inode", "events": [ { "time": "2024-04-15T15:25:31.373265+0000", "event": "initiated" }, { "time": "2024-04-15T15:25:31.373265+0000", "event": "header_read" }, { "time": "2024-04-15T15:25:31.373265+0000", "event": "throttled" }, { "time": "2024-04-15T15:25:31.373265+0000", "event": "all_read" }, { "time": "2024-04-15T15:25:31.373265+0000", "event": "dispatched" }, { "time": "2024-04-15T15:25:31.373473+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:25:31.374435+0000", "event": "failed to xlock, waiting" }, { "time": "2024-04-15T15:25:31.377141+0000", "event": "acquired locks" }, { "time": "2024-04-15T15:25:31.377142+0000", "event": "quiesce blocked" } ], "locks": [] } } } 2024-04-15T15:25:33.512 INFO:tasks.cephfs.test_quiesce:making archive directory /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/quiesce 2024-04-15T15:25:33.514 ERROR:tasks.cephfs.test_quiesce:cache written to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/quiesce/cachesljq_hun 2024-04-15T15:25:33.516 ERROR:tasks.cephfs.test_quiesce:ops written to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/quiesce/opsftxe4yi1 2024-04-15T15:25:33.517 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:25:33.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.730+0000 7f4bd2000640 1 Processor -- start 2024-04-15T15:25:33.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.730+0000 7f4bd2000640 1 -- start start 2024-04-15T15:25:33.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bd2000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc15b410 0x7f4bcc15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:33.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bd2000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4bcc0cf530 0x7f4bcc0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:33.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bd2000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4bcc15f110 0x7f4bcc15f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:33.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bd2000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f4bcc15cae0 con 0x7f4bcc0cf530 2024-04-15T15:25:33.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bd2000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4bcc0cd360 con 0x7f4bcc15f110 2024-04-15T15:25:33.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bd2000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4bcc0cd640 con 0x7f4bcc15b410 2024-04-15T15:25:33.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bcbe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4bcc0cf530 0x7f4bcc0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:33.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bcbe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4bcc0cf530 0x7f4bcc0cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55776/0 (socket says 172.21.15.150:55776) 2024-04-15T15:25:33.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bcbe00640 1 -- 172.21.15.150:0/2470157112 learned_addr learned my addr 172.21.15.150:0/2470157112 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:33.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bcbe00640 1 -- 172.21.15.150:0/2470157112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc15b410 msgr2=0x7f4bcc15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:33.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bd1600640 1 --2- 172.21.15.150:0/2470157112 >> v2:172.21.15.187:3300/0 conn(0x7f4bcc15f110 0x7f4bcc15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:33.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bcbe00640 1 --2- 172.21.15.150:0/2470157112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc15b410 0x7f4bcc15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bcbe00640 1 -- 172.21.15.150:0/2470157112 >> v2:172.21.15.187:3300/0 conn(0x7f4bcc15f110 msgr2=0x7f4bcc15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:33.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bcbe00640 1 --2- 172.21.15.150:0/2470157112 >> v2:172.21.15.187:3300/0 conn(0x7f4bcc15f110 0x7f4bcc15f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.732+0000 7f4bcbe00640 1 -- 172.21.15.150:0/2470157112 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc0cd920 con 0x7f4bcc0cf530 2024-04-15T15:25:33.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.734+0000 7f4bcbe00640 1 --2- 172.21.15.150:0/2470157112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc0cf530 0x7f4bcc0cf9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0b3020 tx=0x7f4bb802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419f79851349795b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:33.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.734+0000 7f4bcb400640 1 -- 172.21.15.150:0/2470157112 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80317d0 con 0x7f4bcc0cf530 2024-04-15T15:25:33.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.734+0000 7f4bcb400640 1 -- 172.21.15.150:0/2470157112 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb8037070 con 0x7f4bcc0cf530 2024-04-15T15:25:33.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.734+0000 7f4bcb400640 1 -- 172.21.15.150:0/2470157112 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8031b90 con 0x7f4bcc0cf530 2024-04-15T15:25:33.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.734+0000 7f4bd2000640 1 -- 172.21.15.150:0/2470157112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc0cf530 msgr2=0x7f4bcc0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:33.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.734+0000 7f4bd2000640 1 --2- 172.21.15.150:0/2470157112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc0cf530 0x7f4bcc0cf9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0b3020 tx=0x7f4bb802b870 comp rx=0 tx=0).stop 2024-04-15T15:25:33.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.735+0000 7f4bd2000640 1 -- 172.21.15.150:0/2470157112 shutdown_connections 2024-04-15T15:25:33.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.735+0000 7f4bd2000640 1 --2- 172.21.15.150:0/2470157112 >> v2:172.21.15.187:3300/0 conn(0x7f4bcc15f110 0x7f4bcc15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.735+0000 7f4bd2000640 1 --2- 172.21.15.150:0/2470157112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc0cf530 0x7f4bcc0cf9c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.735+0000 7f4bd2000640 1 --2- 172.21.15.150:0/2470157112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc15b410 0x7f4bcc15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.735+0000 7f4bd2000640 1 -- 172.21.15.150:0/2470157112 >> 172.21.15.150:0/2470157112 conn(0x7f4bcc0b23a0 msgr2=0x7f4bcc0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:33.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.735+0000 7f4bd2000640 1 -- 172.21.15.150:0/2470157112 shutdown_connections 2024-04-15T15:25:33.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.735+0000 7f4bd2000640 1 -- 172.21.15.150:0/2470157112 wait complete. 2024-04-15T15:25:33.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.736+0000 7f4bd2000640 1 Processor -- start 2024-04-15T15:25:33.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.737+0000 7f4bd2000640 1 -- start start 2024-04-15T15:25:33.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.737+0000 7f4bd2000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc0cf530 0x7f4bcc14e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:33.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.737+0000 7f4bd2000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bcc15b410 0x7f4bcc155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:33.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.737+0000 7f4bd2000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc15f110 0x7f4bcc071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:33.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.737+0000 7f4bd2000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc0cff00 con 0x7f4bcc15f110 2024-04-15T15:25:33.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.737+0000 7f4bd2000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc0cdee0 con 0x7f4bcc15b410 2024-04-15T15:25:33.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.737+0000 7f4bd2000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4bcc0cd030 con 0x7f4bcc0cf530 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd1600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc15f110 0x7f4bcc071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd1600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc15f110 0x7f4bcc071d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55782/0 (socket says 172.21.15.150:55782) 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd1600640 1 -- 172.21.15.150:0/4238922555 learned_addr learned my addr 172.21.15.150:0/4238922555 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd0c00640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc0cf530 0x7f4bcc14e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bcbe00640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bcc15b410 0x7f4bcc155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd1600640 1 -- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc0cf530 msgr2=0x7f4bcc14e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd1600640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc0cf530 0x7f4bcc14e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd1600640 1 -- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bcc15b410 msgr2=0x7f4bcc155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd1600640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bcc15b410 0x7f4bcc155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.738+0000 7f4bd1600640 1 -- 172.21.15.150:0/4238922555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc14e560 con 0x7f4bcc15f110 2024-04-15T15:25:33.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.739+0000 7f4bd1600640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc15f110 0x7f4bcc071d30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400cc30 tx=0x7f4bc400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:33.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.739+0000 7f4bc8c00640 1 -- 172.21.15.150:0/4238922555 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400ce40 con 0x7f4bcc15f110 2024-04-15T15:25:33.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.739+0000 7f4bc8c00640 1 -- 172.21.15.150:0/4238922555 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4003990 con 0x7f4bcc15f110 2024-04-15T15:25:33.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.739+0000 7f4bc8c00640 1 -- 172.21.15.150:0/4238922555 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4003d40 con 0x7f4bcc15f110 2024-04-15T15:25:33.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.739+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc0b3490 con 0x7f4bcc15f110 2024-04-15T15:25:33.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.739+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc156da0 con 0x7f4bcc15f110 2024-04-15T15:25:33.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.740+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bcc0755c0 con 0x7f4bcc15f110 2024-04-15T15:25:33.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.741+0000 7f4bc8c00640 1 -- 172.21.15.150:0/4238922555 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4bc4003ee0 con 0x7f4bcc15f110 2024-04-15T15:25:33.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.742+0000 7f4bc8c00640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4bac06c980 0x7f4bac06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:33.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.742+0000 7f4bc8c00640 1 -- 172.21.15.150:0/4238922555 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f4bc408c2a0 con 0x7f4bcc15f110 2024-04-15T15:25:33.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.743+0000 7f4bd0c00640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4bac06c980 0x7f4bac06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:33.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.744+0000 7f4bd0c00640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4bac06c980 0x7f4bac06ee40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009890 tx=0x7f4bc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:33.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.747+0000 7f4bc8c00640 1 -- 172.21.15.150:0/4238922555 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bcc14e560 con 0x7f4bcc15f110 2024-04-15T15:25:33.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.988+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4bcc1515a0 con 0x7f4bcc15f110 2024-04-15T15:25:33.991 INFO:tasks.ceph.mon.a.smithi150.stderr:2024-04-15T15:25:33.989+0000 7f8c07200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:25:33.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.992+0000 7f4bc8c00640 1 -- 172.21.15.150:0/4238922555 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f4bc4014030 con 0x7f4bcc15f110 2024-04-15T15:25:33.994 INFO:teuthology.orchestra.run.smithi150.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4bac06c980 msgr2=0x7f4bac06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4bac06c980 0x7f4bac06ee40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009890 tx=0x7f4bc0008040 comp rx=0 tx=0).stop 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc15f110 msgr2=0x7f4bcc071d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc15f110 0x7f4bcc071d30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400cc30 tx=0x7f4bc400a5c0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 shutdown_connections 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bcc15f110 0x7f4bcc071d30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bcc15b410 0x7f4bcc155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4bac06c980 0x7f4bac06ee40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 --2- 172.21.15.150:0/4238922555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bcc0cf530 0x7f4bcc14e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:33.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.997+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 >> 172.21.15.150:0/4238922555 conn(0x7f4bcc0b23a0 msgr2=0x7f4bcc15f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:34.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.998+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 shutdown_connections 2024-04-15T15:25:34.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:33.998+0000 7f4bd2000640 1 -- 172.21.15.150:0/4238922555 wait complete. 2024-04-15T15:25:34.021 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-15T15:25:34.072 INFO:tasks.ceph.mds.a.smithi150.stdout:starting mds.a at 2024-04-15T15:25:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.213+0000 7f8b43e00640 1 Processor -- start 2024-04-15T15:25:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.213+0000 7f8b43e00640 1 -- start start 2024-04-15T15:25:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.214+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b44162730 0x7f8b440b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.214+0000 7f8b43e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8b440d2b50 0x7f8b440d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.214+0000 7f8b43e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b44162dd0 con 0x7f8b4415c680 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.214+0000 7f8b43e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f8b440d34f0 con 0x7f8b440d2b50 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.214+0000 7f8b43e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8b44162b90 con 0x7f8b44162730 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.215+0000 7f8b43400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8b440d2b50 0x7f8b440d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.215+0000 7f8b43400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8b440d2b50 0x7f8b440d2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55822/0 (socket says 172.21.15.150:55822) 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.215+0000 7f8b43400640 1 -- 172.21.15.150:0/1083170683 learned_addr learned my addr 172.21.15.150:0/1083170683 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.215+0000 7f8b43400640 1 -- 172.21.15.150:0/1083170683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b44162730 msgr2=0x7f8b440b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.215+0000 7f8b43400640 1 --2- 172.21.15.150:0/1083170683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b44162730 0x7f8b440b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.215+0000 7f8b43400640 1 -- 172.21.15.150:0/1083170683 >> v1:172.21.15.187:6789/0 conn(0x7f8b4415c680 legacy=0x7f8b4415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.215+0000 7f8b43400640 1 -- 172.21.15.150:0/1083170683 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b441630b0 con 0x7f8b440d2b50 2024-04-15T15:25:34.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.216+0000 7f8b43400640 1 --2- 172.21.15.150:0/1083170683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b440d2fb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009e50 tx=0x7f8b3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f1a0c7b11f3c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:34.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.216+0000 7f8b41600640 1 -- 172.21.15.150:0/1083170683 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c02ca70 con 0x7f8b440d2b50 2024-04-15T15:25:34.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.216+0000 7f8b41600640 1 -- 172.21.15.150:0/1083170683 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3c02cc10 con 0x7f8b440d2b50 2024-04-15T15:25:34.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.216+0000 7f8b41600640 1 -- 172.21.15.150:0/1083170683 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0343f0 con 0x7f8b440d2b50 2024-04-15T15:25:34.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.217+0000 7f8b43e00640 1 -- 172.21.15.150:0/1083170683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 msgr2=0x7f8b440d2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:34.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.217+0000 7f8b43e00640 1 --2- 172.21.15.150:0/1083170683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b440d2fb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009e50 tx=0x7f8b3c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.217+0000 7f8b43e00640 1 -- 172.21.15.150:0/1083170683 shutdown_connections 2024-04-15T15:25:34.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.217+0000 7f8b43e00640 1 --2- 172.21.15.150:0/1083170683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b440d2fb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.217+0000 7f8b43e00640 1 --2- 172.21.15.150:0/1083170683 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b44162730 0x7f8b440b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.217+0000 7f8b43e00640 1 -- 172.21.15.150:0/1083170683 >> 172.21.15.150:0/1083170683 conn(0x7f8b44073480 msgr2=0x7f8b44073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:34.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.217+0000 7f8b43e00640 1 -- 172.21.15.150:0/1083170683 shutdown_connections 2024-04-15T15:25:34.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.217+0000 7f8b43e00640 1 -- 172.21.15.150:0/1083170683 wait complete. 2024-04-15T15:25:34.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.219+0000 7f8b43e00640 1 Processor -- start 2024-04-15T15:25:34.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.219+0000 7f8b43e00640 1 -- start start 2024-04-15T15:25:34.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b441e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:34.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b4415c680 0x7f8b440c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:34.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b44162730 0x7f8b4415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:34.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b43e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4415ca60 con 0x7f8b440d2b50 2024-04-15T15:25:34.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b43e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4415f330 con 0x7f8b4415c680 2024-04-15T15:25:34.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b43e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4415ff60 con 0x7f8b44162730 2024-04-15T15:25:34.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b42a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b441e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b42a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b441e98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55826/0 (socket says 172.21.15.150:55826) 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.220+0000 7f8b42a00640 1 -- 172.21.15.150:0/2978124467 learned_addr learned my addr 172.21.15.150:0/2978124467 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b42a00640 1 -- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b44162730 msgr2=0x7f8b4415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b42a00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b44162730 0x7f8b4415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b42a00640 1 -- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b4415c680 msgr2=0x7f8b440c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b42a00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b4415c680 0x7f8b440c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b42a00640 1 -- 172.21.15.150:0/2978124467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b441f4910 con 0x7f8b440d2b50 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b42a00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b441e98a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400c410 tx=0x7f8b3400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b3b400640 1 -- 172.21.15.150:0/2978124467 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34017070 con 0x7f8b440d2b50 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b3b400640 1 -- 172.21.15.150:0/2978124467 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b340035f0 con 0x7f8b440d2b50 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.221+0000 7f8b3b400640 1 -- 172.21.15.150:0/2978124467 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34003a20 con 0x7f8b440d2b50 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.222+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b440ca710 con 0x7f8b440d2b50 2024-04-15T15:25:34.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.222+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b440cacd0 con 0x7f8b440d2b50 2024-04-15T15:25:34.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.223+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b440755c0 con 0x7f8b440d2b50 2024-04-15T15:25:34.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.224+0000 7f8b3b400640 1 -- 172.21.15.150:0/2978124467 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8b3401a070 con 0x7f8b440d2b50 2024-04-15T15:25:34.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.225+0000 7f8b3b400640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8b2406c980 0x7f8b2406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:34.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.225+0000 7f8b3b400640 1 -- 172.21.15.150:0/2978124467 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f8b3408b120 con 0x7f8b440d2b50 2024-04-15T15:25:34.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.225+0000 7f8b42000640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8b2406c980 0x7f8b2406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:34.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.227+0000 7f8b42000640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8b2406c980 0x7f8b2406ee40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8b440d44a0 tx=0x7f8b30006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:34.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.229+0000 7f8b3b400640 1 -- 172.21.15.150:0/2978124467 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b34059490 con 0x7f8b440d2b50 2024-04-15T15:25:34.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.471+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8b4406dbe0 con 0x7f8b440d2b50 2024-04-15T15:25:34.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.883+0000 7f8b3b400640 1 -- 172.21.15.150:0/2978124467 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v19) ==== 162+0+0 (secure 0 0 0) 0x7f8b340010c0 con 0x7f8b440d2b50 2024-04-15T15:25:34.885 INFO:teuthology.orchestra.run.smithi150.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-15T15:25:34.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8b2406c980 msgr2=0x7f8b2406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:34.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8b2406c980 0x7f8b2406ee40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8b440d44a0 tx=0x7f8b30006bf0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 msgr2=0x7f8b441e98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b441e98a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400c410 tx=0x7f8b3400c8d0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 shutdown_connections 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8b2406c980 0x7f8b2406ee40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b44162730 0x7f8b4415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b4415c680 0x7f8b440c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 --2- 172.21.15.150:0/2978124467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b440d2b50 0x7f8b441e98a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.887+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 >> 172.21.15.150:0/2978124467 conn(0x7f8b44073480 msgr2=0x7f8b440d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.888+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 shutdown_connections 2024-04-15T15:25:34.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:34.888+0000 7f8b43e00640 1 -- 172.21.15.150:0/2978124467 wait complete. 2024-04-15T15:25:34.913 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:35.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.111+0000 7f1957e00640 1 Processor -- start 2024-04-15T15:25:35.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.111+0000 7f1957e00640 1 -- start start 2024-04-15T15:25:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.113+0000 7f1957e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f195015ee00 0x7f19500cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:35.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.113+0000 7f1957e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19500cf6f0 0x7f19500b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:35.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.113+0000 7f1957e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19500cfdc0 con 0x7f19500d3e20 2024-04-15T15:25:35.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.113+0000 7f1957e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f19500cfba0 con 0x7f195015ee00 2024-04-15T15:25:35.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.113+0000 7f1957e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19500d2b10 con 0x7f19500cf6f0 2024-04-15T15:25:35.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.114+0000 7f1955e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19500cf6f0 0x7f19500b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:35.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.114+0000 7f1954a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f195015ee00 0x7f19500cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:35.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.114+0000 7f1955400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f19500d3e20 0x7f19500d4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:51010/0 (socket says 172.21.15.150:51010) 2024-04-15T15:25:35.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.114+0000 7f1955400640 1 -- 172.21.15.150:0/1248312030 learned_addr learned my addr 172.21.15.150:0/1248312030 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:35.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.115+0000 7f1954a00640 1 -- 172.21.15.150:0/1248312030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19500cf6f0 msgr2=0x7f19500b79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:35.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.115+0000 7f1954a00640 1 --2- 172.21.15.150:0/1248312030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19500cf6f0 0x7f19500b79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.115+0000 7f1954a00640 1 -- 172.21.15.150:0/1248312030 >> v1:172.21.15.187:6789/0 conn(0x7f19500d3e20 legacy=0x7f19500d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:35.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.115+0000 7f1954a00640 1 -- 172.21.15.150:0/1248312030 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19500d2d60 con 0x7f195015ee00 2024-04-15T15:25:35.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.116+0000 7f1954a00640 1 --2- 172.21.15.150:0/1248312030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f195015ee00 0x7f19500cf1b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f193c0097a0 tx=0x7f193c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b984bae68c1907e server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:35.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f194fe00640 1 -- 172.21.15.150:0/1248312030 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c034070 con 0x7f195015ee00 2024-04-15T15:25:35.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f194fe00640 1 -- 172.21.15.150:0/1248312030 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c030610 con 0x7f195015ee00 2024-04-15T15:25:35.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f194fe00640 1 -- 172.21.15.150:0/1248312030 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0309e0 con 0x7f195015ee00 2024-04-15T15:25:35.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f1957e00640 1 -- 172.21.15.150:0/1248312030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f195015ee00 msgr2=0x7f19500cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:35.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f1957e00640 1 --2- 172.21.15.150:0/1248312030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f195015ee00 0x7f19500cf1b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f193c0097a0 tx=0x7f193c02b960 comp rx=0 tx=0).stop 2024-04-15T15:25:35.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f1957e00640 1 -- 172.21.15.150:0/1248312030 shutdown_connections 2024-04-15T15:25:35.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f1957e00640 1 --2- 172.21.15.150:0/1248312030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19500cf6f0 0x7f19500b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f1957e00640 1 --2- 172.21.15.150:0/1248312030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f195015ee00 0x7f19500cf1b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f1957e00640 1 -- 172.21.15.150:0/1248312030 >> 172.21.15.150:0/1248312030 conn(0x7f195006d470 msgr2=0x7f195006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:35.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f1957e00640 1 -- 172.21.15.150:0/1248312030 shutdown_connections 2024-04-15T15:25:35.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.117+0000 7f1957e00640 1 -- 172.21.15.150:0/1248312030 wait complete. 2024-04-15T15:25:35.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.119+0000 7f1957e00640 1 Processor -- start 2024-04-15T15:25:35.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.119+0000 7f1957e00640 1 -- start start 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1957e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19500cf6f0 0x7f195014ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1957e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19500d3e20 0x7f19500cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1957e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f195015ee00 0x7f19501f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1957e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f19500d3840 con 0x7f19500d3e20 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1957e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f19500d33b0 con 0x7f19500cf6f0 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1957e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19500d39c0 con 0x7f195015ee00 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1954a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19500d3e20 0x7f19500cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1954a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19500d3e20 0x7f19500cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55842/0 (socket says 172.21.15.150:55842) 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.120+0000 7f1954a00640 1 -- 172.21.15.150:0/3924822901 learned_addr learned my addr 172.21.15.150:0/3924822901 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1955400640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19500cf6f0 0x7f195014ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1955e00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f195015ee00 0x7f19501f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1954a00640 1 -- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f195015ee00 msgr2=0x7f19501f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:35.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1954a00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f195015ee00 0x7f19501f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1954a00640 1 -- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19500cf6f0 msgr2=0x7f195014ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:35.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1954a00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19500cf6f0 0x7f195014ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1954a00640 1 -- 172.21.15.150:0/3924822901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19500b54a0 con 0x7f19500d3e20 2024-04-15T15:25:35.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1955400640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19500cf6f0 0x7f195014ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:25:35.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.121+0000 7f1954a00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19500d3e20 0x7f19500cac50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f193c0098c0 tx=0x7f193c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:35.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.122+0000 7f194d600640 1 -- 172.21.15.150:0/3924822901 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c034040 con 0x7f19500d3e20 2024-04-15T15:25:35.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.122+0000 7f194d600640 1 -- 172.21.15.150:0/3924822901 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c03aaa0 con 0x7f19500d3e20 2024-04-15T15:25:35.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.122+0000 7f194d600640 1 -- 172.21.15.150:0/3924822901 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0443f0 con 0x7f19500d3e20 2024-04-15T15:25:35.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.122+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19500b5780 con 0x7f19500d3e20 2024-04-15T15:25:35.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.122+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19500b5d40 con 0x7f19500d3e20 2024-04-15T15:25:35.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.123+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1950068c80 con 0x7f19500d3e20 2024-04-15T15:25:35.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.124+0000 7f194d600640 1 -- 172.21.15.150:0/3924822901 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f193c04b060 con 0x7f19500d3e20 2024-04-15T15:25:35.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.125+0000 7f194d600640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f193006c980 0x7f193006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:35.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.125+0000 7f194d600640 1 -- 172.21.15.150:0/3924822901 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f193c0adaf0 con 0x7f19500d3e20 2024-04-15T15:25:35.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.126+0000 7f1955400640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f193006c980 0x7f193006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:35.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.127+0000 7f1955400640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f193006c980 0x7f193006ee40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19440012e0 tx=0x7f1944009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:35.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.130+0000 7f194d600640 1 -- 172.21.15.150:0/3924822901 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f193c03f070 con 0x7f19500d3e20 2024-04-15T15:25:35.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.370+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f195006dcb0 con 0x7f19500d3e20 2024-04-15T15:25:35.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.371+0000 7f194d600640 1 -- 172.21.15.150:0/3924822901 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5187 (secure 0 0 0) 0x7f193c04b350 con 0x7f19500d3e20 2024-04-15T15:25:35.373 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:35.373 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:34.880408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5499":{"gid":5499,"name":"a","rank":0,"incarnation":19,"state":"up:replay","state_seq":1,"addr":"172.21.15.150:6835/3243584315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3243584315},{"type":"v1","addr":"172.21.15.150:6835","nonce":3243584315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-15T15:25:35.374 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 19 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f193006c980 msgr2=0x7f193006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f193006c980 0x7f193006ee40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19440012e0 tx=0x7f1944009040 comp rx=0 tx=0).stop 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19500d3e20 msgr2=0x7f19500cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19500d3e20 0x7f19500cac50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f193c0098c0 tx=0x7f193c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 shutdown_connections 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f193006c980 0x7f193006ee40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f195015ee00 0x7f19501f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19500d3e20 0x7f19500cac50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 --2- 172.21.15.150:0/3924822901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19500cf6f0 0x7f195014ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 >> 172.21.15.150:0/3924822901 conn(0x7f195006d470 msgr2=0x7f19500cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:35.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 shutdown_connections 2024-04-15T15:25:35.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:35.374+0000 7f1957e00640 1 -- 172.21.15.150:0/3924822901 wait complete. 2024-04-15T15:25:35.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:25:06.343634+0000', 'modified': '2024-04-15T15:25:34.880408+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5499': {'gid': 5499, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/3243584315', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 3243584315}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 3243584315}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-15T15:25:35.798 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5499:up:replay 2024-04-15T15:25:36.798 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:37.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.018+0000 7f1072e00640 1 Processor -- start 2024-04-15T15:25:37.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.018+0000 7f1072e00640 1 -- start start 2024-04-15T15:25:37.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.019+0000 7f1072e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 0x7f106c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:37.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.019+0000 7f1072e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f106c06f940 con 0x7f106c070210 2024-04-15T15:25:37.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.019+0000 7f1072e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f106c06e8a0 con 0x7f106c15b6d0 2024-04-15T15:25:37.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.019+0000 7f1072e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f106c06eb80 con 0x7f106c0b51f0 2024-04-15T15:25:37.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.020+0000 7f1063400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f106c070210 0x7f106c0b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47664/0 (socket says 172.21.15.150:47664) 2024-04-15T15:25:37.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.020+0000 7f1063400640 1 -- 172.21.15.150:0/3075068802 learned_addr learned my addr 172.21.15.150:0/3075068802 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.020+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131712229 0 0) 0x7f106c06f940 con 0x7f106c070210 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.020+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1050003520 con 0x7f106c070210 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f1070e00640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 0x7f106c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2387527304 0 0) 0x7f1050003520 con 0x7f106c070210 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 msgr2=0x7f106c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 0x7f106c15cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 >> v1:172.21.15.187:6789/0 conn(0x7f106c15b6d0 legacy=0x7f106c0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c06ee60 con 0x7f106c070210 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f10540019f0 con 0x7f106c070210 2024-04-15T15:25:37.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 >> v1:172.21.15.150:6789/0 conn(0x7f106c070210 legacy=0x7f106c0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10500039e0 0x7f1050003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 0x7f1050004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1050004e10 0x7f1050005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f106c06eb80 con 0x7f10500039e0 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f105000bd20 con 0x7f1050004e10 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f105000bf20 con 0x7f106c0b51f0 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f106be00640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10500039e0 0x7f1050003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.021+0000 7f1063400640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 0x7f1050004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.022+0000 7f106be00640 1 -- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 msgr2=0x7f1050004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.022+0000 7f1070e00640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1050004e10 0x7f1050005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.022+0000 7f106be00640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 0x7f1050004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.022+0000 7f106be00640 1 -- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1050004e10 msgr2=0x7f1050005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.022+0000 7f106be00640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1050004e10 0x7f1050005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.022+0000 7f106be00640 1 -- 172.21.15.150:0/3075068802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1050003520 con 0x7f10500039e0 2024-04-15T15:25:37.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.022+0000 7f1063400640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106c0b51f0 0x7f1050004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:37.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.022+0000 7f1070e00640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1050004e10 0x7f1050005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:37.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.023+0000 7f106be00640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10500039e0 0x7f1050003e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f105c001330 tx=0x7f1050005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc54f66f2ae0c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:37.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.023+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c001370 con 0x7f10500039e0 2024-04-15T15:25:37.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.023+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105c02f040 con 0x7f10500039e0 2024-04-15T15:25:37.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.023+0000 7f106b400640 1 -- 172.21.15.150:0/3075068802 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0035f0 con 0x7f10500039e0 2024-04-15T15:25:37.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.023+0000 7f1072e00640 1 -- 172.21.15.150:0/3075068802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10500039e0 msgr2=0x7f1050003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:37.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.023+0000 7f1072e00640 1 --2- 172.21.15.150:0/3075068802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10500039e0 0x7f1050003e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f105c001330 tx=0x7f1050005830 comp rx=0 tx=0).stop 2024-04-15T15:25:37.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.023+0000 7f1070e00640 1 -- 172.21.15.150:0/3075068802 reap_dead start 2024-04-15T15:25:37.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.024+0000 7f1072e00640 1 -- 172.21.15.150:0/3075068802 shutdown_connections 2024-04-15T15:25:37.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.024+0000 7f1072e00640 1 -- 172.21.15.150:0/3075068802 >> 172.21.15.150:0/3075068802 conn(0x7f106c0b27b0 msgr2=0x7f106c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:37.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.024+0000 7f1072e00640 1 -- 172.21.15.150:0/3075068802 shutdown_connections 2024-04-15T15:25:37.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.024+0000 7f1072e00640 1 -- 172.21.15.150:0/3075068802 wait complete. 2024-04-15T15:25:37.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.026+0000 7f1072e00640 1 Processor -- start 2024-04-15T15:25:37.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.026+0000 7f1072e00640 1 -- start start 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1072e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10500039e0 0x7f106c14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1072e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f106c0af000 0x7f106c1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1072e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106c1f6120 0x7f106c1e6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1072e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f106c0ce420 con 0x7f106c0af000 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1072e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f106c06f420 con 0x7f106c1f6120 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1072e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f106c15cac0 con 0x7f10500039e0 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1063400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f106c0af000 0x7f106c1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1063400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f106c0af000 0x7f106c1f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36066/0 (socket says 172.21.15.150:36066) 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1063400640 1 -- 172.21.15.150:0/3507805403 learned_addr learned my addr 172.21.15.150:0/3507805403 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:37.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1063400640 1 -- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10500039e0 msgr2=0x7f106c14f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1063400640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10500039e0 0x7f106c14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1063400640 1 -- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106c1f6120 msgr2=0x7f106c1e6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1063400640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106c1f6120 0x7f106c1e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.027+0000 7f1063400640 1 -- 172.21.15.150:0/3507805403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c153940 con 0x7f106c0af000 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.028+0000 7f1063400640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f106c0af000 0x7f106c1f56b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f10540021a0 tx=0x7f105400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.028+0000 7f1068c00640 1 -- 172.21.15.150:0/3507805403 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054019070 con 0x7f106c0af000 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.028+0000 7f1068c00640 1 -- 172.21.15.150:0/3507805403 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1054004d60 con 0x7f106c0af000 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.028+0000 7f1068c00640 1 -- 172.21.15.150:0/3507805403 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054007310 con 0x7f106c0af000 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.028+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c153bb0 con 0x7f106c0af000 2024-04-15T15:25:37.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.028+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c154170 con 0x7f106c0af000 2024-04-15T15:25:37.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.029+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106c0b2d00 con 0x7f106c0af000 2024-04-15T15:25:37.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.030+0000 7f1068c00640 1 -- 172.21.15.150:0/3507805403 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f10540074b0 con 0x7f106c0af000 2024-04-15T15:25:37.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.032+0000 7f1068c00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f104806c980 0x7f104806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:37.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.032+0000 7f1068c00640 1 -- 172.21.15.150:0/3507805403 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f105408bda0 con 0x7f106c0af000 2024-04-15T15:25:37.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.032+0000 7f106be00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f104806c980 0x7f104806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:37.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.034+0000 7f106be00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f104806c980 0x7f104806ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f105c036000 tx=0x7f105c032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:37.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.037+0000 7f1068c00640 1 -- 172.21.15.150:0/3507805403 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f106c0b2d00 con 0x7f106c0af000 2024-04-15T15:25:37.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.276+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f106c068c80 con 0x7f106c0af000 2024-04-15T15:25:37.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.277+0000 7f1068c00640 1 -- 172.21.15.150:0/3507805403 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5187 (secure 0 0 0) 0x7f10540599b0 con 0x7f106c0af000 2024-04-15T15:25:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:37.279 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:35.918715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5499":{"gid":5499,"name":"a","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.150:6835/3243584315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3243584315},{"type":"v1","addr":"172.21.15.150:6835","nonce":3243584315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-15T15:25:37.279 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 21 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f104806c980 msgr2=0x7f104806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f104806c980 0x7f104806ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f105c036000 tx=0x7f105c032000 comp rx=0 tx=0).stop 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f106c0af000 msgr2=0x7f106c1f56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f106c0af000 0x7f106c1f56b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f10540021a0 tx=0x7f105400b560 comp rx=0 tx=0).stop 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 shutdown_connections 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106c1f6120 0x7f106c1e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f106c0af000 0x7f106c1f56b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f104806c980 0x7f104806ee40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 --2- 172.21.15.150:0/3507805403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10500039e0 0x7f106c14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 >> 172.21.15.150:0/3507805403 conn(0x7f106c0b27b0 msgr2=0x7f106c15a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 shutdown_connections 2024-04-15T15:25:37.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:37.280+0000 7f1072e00640 1 -- 172.21.15.150:0/3507805403 wait complete. 2024-04-15T15:25:37.703 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:25:06.343634+0000', 'modified': '2024-04-15T15:25:35.918715+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5499': {'gid': 5499, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.150:6835/3243584315', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 3243584315}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 3243584315}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-15T15:25:37.704 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5499:up:rejoin 2024-04-15T15:25:38.704 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:38.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.921+0000 7fa2cc800640 1 Processor -- start 2024-04-15T15:25:38.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.921+0000 7fa2cc800640 1 -- start start 2024-04-15T15:25:38.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.923+0000 7fa2cc800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa2c40b0060 0x7fa2c406fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:38.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.923+0000 7fa2cc800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c406f460 con 0x7fa2c4162db0 2024-04-15T15:25:38.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.923+0000 7fa2cc800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c40b4100 con 0x7fa2c4070160 2024-04-15T15:25:38.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.923+0000 7fa2cc800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c406f790 con 0x7fa2c40b0060 2024-04-15T15:25:38.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.923+0000 7fa2c9e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa2c4162db0 0x7fa2c40b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47678/0 (socket says 172.21.15.150:47678) 2024-04-15T15:25:38.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.923+0000 7fa2c9e00640 1 -- 172.21.15.150:0/2815217405 learned_addr learned my addr 172.21.15.150:0/2815217405 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064744451 0 0) 0x7fa2c406f460 con 0x7fa2c4162db0 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2ac003520 con 0x7fa2c4162db0 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c9400640 1 --2- 172.21.15.150:0/2815217405 >> v2:172.21.15.187:3300/0 conn(0x7fa2c40b0060 0x7fa2c406fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2786604579 0 0) 0x7fa2ac003520 con 0x7fa2c4162db0 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 >> v2:172.21.15.187:3300/0 conn(0x7fa2c40b0060 msgr2=0x7fa2c406fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 --2- 172.21.15.150:0/2815217405 >> v2:172.21.15.187:3300/0 conn(0x7fa2c40b0060 0x7fa2c406fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 >> v1:172.21.15.187:6789/0 conn(0x7fa2c4070160 legacy=0x7fa2c40ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c40b43e0 con 0x7fa2c4162db0 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fa2b8001670 con 0x7fa2c4162db0 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 >> v1:172.21.15.150:6789/0 conn(0x7fa2c4162db0 legacy=0x7fa2c40b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2ac0039b0 0x7fa2ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2ac004800 0x7fa2ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:38.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c406f790 con 0x7fa2ac0051c0 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac00c060 con 0x7fa2ac004800 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.924+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2ac00c260 con 0x7fa2ac0039b0 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.925+0000 7fa2ca800640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.925+0000 7fa2c9e00640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2ac0039b0 0x7fa2ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.925+0000 7fa2ca800640 1 -- 172.21.15.150:0/2815217405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2ac0039b0 msgr2=0x7fa2ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.925+0000 7fa2ca800640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2ac0039b0 0x7fa2ac003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.925+0000 7fa2ca800640 1 -- 172.21.15.150:0/2815217405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2ac004800 msgr2=0x7fa2ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.925+0000 7fa2c9400640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2ac004800 0x7fa2ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.925+0000 7fa2ca800640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2ac004800 0x7fa2ac004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:38.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.925+0000 7fa2ca800640 1 -- 172.21.15.150:0/2815217405 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac003520 con 0x7fa2ac0051c0 2024-04-15T15:25:38.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.926+0000 7fa2ca800640 1 -- 172.21.15.150:0/2815217405 reap_dead start 2024-04-15T15:25:38.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.926+0000 7fa2ca800640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2ac00b9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0001e50 tx=0x7fa2ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d0762ce2bbee9c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:38.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.927+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0001a70 con 0x7fa2ac0051c0 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.927+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0003bb0 con 0x7fa2ac0051c0 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.927+0000 7fa2c8a00640 1 -- 172.21.15.150:0/2815217405 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00050d0 con 0x7fa2ac0051c0 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.927+0000 7fa2cc800640 1 -- 172.21.15.150:0/2815217405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 msgr2=0x7fa2ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.927+0000 7fa2cc800640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2ac00b9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0001e50 tx=0x7fa2ac005830 comp rx=0 tx=0).stop 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.928+0000 7fa2cc800640 1 -- 172.21.15.150:0/2815217405 shutdown_connections 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.928+0000 7fa2cc800640 1 --2- 172.21.15.150:0/2815217405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2ac00b9c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.928+0000 7fa2cc800640 1 -- 172.21.15.150:0/2815217405 >> 172.21.15.150:0/2815217405 conn(0x7fa2c40b27b0 msgr2=0x7fa2c40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.928+0000 7fa2cc800640 1 -- 172.21.15.150:0/2815217405 shutdown_connections 2024-04-15T15:25:38.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.928+0000 7fa2cc800640 1 -- 172.21.15.150:0/2815217405 wait complete. 2024-04-15T15:25:38.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.930+0000 7fa2cc800640 1 Processor -- start 2024-04-15T15:25:38.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.930+0000 7fa2cc800640 1 -- start start 2024-04-15T15:25:38.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2cc800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2c415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:38.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2cc800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2c4165ff0 0x7fa2c414c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:38.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2cc800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2c4156e00 0x7fa2c4158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:38.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2cc800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4163310 con 0x7fa2ac0051c0 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2cc800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4163190 con 0x7fa2c4156e00 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2cc800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c406fc20 con 0x7fa2c4165ff0 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2c9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2c415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2c9e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2c415a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36074/0 (socket says 172.21.15.150:36074) 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.931+0000 7fa2c9e00640 1 -- 172.21.15.150:0/4248161255 learned_addr learned my addr 172.21.15.150:0/4248161255 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2c9e00640 1 -- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2c4165ff0 msgr2=0x7fa2c414c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2c9e00640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2c4165ff0 0x7fa2c414c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2c9e00640 1 -- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2c4156e00 msgr2=0x7fa2c4158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2c9e00640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2c4156e00 0x7fa2c4158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2c9e00640 1 -- 172.21.15.150:0/4248161255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c415a960 con 0x7fa2ac0051c0 2024-04-15T15:25:38.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2c9e00640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2c415a420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800d770 tx=0x7fa2b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:38.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2be000640 1 -- 172.21.15.150:0/4248161255 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8019070 con 0x7fa2ac0051c0 2024-04-15T15:25:38.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2be000640 1 -- 172.21.15.150:0/4248161255 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b80074e0 con 0x7fa2ac0051c0 2024-04-15T15:25:38.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2be000640 1 -- 172.21.15.150:0/4248161255 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b80049e0 con 0x7fa2ac0051c0 2024-04-15T15:25:38.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.932+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c4157730 con 0x7fa2ac0051c0 2024-04-15T15:25:38.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.933+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c4163810 con 0x7fa2ac0051c0 2024-04-15T15:25:38.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.934+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c4068c80 con 0x7fa2ac0051c0 2024-04-15T15:25:38.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.935+0000 7fa2be000640 1 -- 172.21.15.150:0/4248161255 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa2b8004b80 con 0x7fa2ac0051c0 2024-04-15T15:25:38.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.936+0000 7fa2be000640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa2a806c980 0x7fa2a806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:38.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.937+0000 7fa2be000640 1 -- 172.21.15.150:0/4248161255 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fa2b808b670 con 0x7fa2ac0051c0 2024-04-15T15:25:38.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.937+0000 7fa2c9400640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa2a806c980 0x7fa2a806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:38.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.939+0000 7fa2c9400640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa2a806c980 0x7fa2a806ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2c415ced0 tx=0x7fa2b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:38.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:38.942+0000 7fa2be000640 1 -- 172.21.15.150:0/4248161255 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2b80598d0 con 0x7fa2ac0051c0 2024-04-15T15:25:39.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.184+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2c40cf730 con 0x7fa2ac0051c0 2024-04-15T15:25:39.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.185+0000 7fa2be000640 1 -- 172.21.15.150:0/4248161255 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5194 (secure 0 0 0) 0x7fa2b80591d0 con 0x7fa2ac0051c0 2024-04-15T15:25:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:39.187 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:37.899562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5499":{"gid":5499,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6835/3243584315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3243584315},{"type":"v1","addr":"172.21.15.150:6835","nonce":3243584315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5499,"qdb_cluster":[5499]},"id":2}]} 2024-04-15T15:25:39.188 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 22 2024-04-15T15:25:39.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa2a806c980 msgr2=0x7fa2a806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa2a806c980 0x7fa2a806ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2c415ced0 tx=0x7fa2b4008040 comp rx=0 tx=0).stop 2024-04-15T15:25:39.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 msgr2=0x7fa2c415a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2c415a420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800d770 tx=0x7fa2b800dc30 comp rx=0 tx=0).stop 2024-04-15T15:25:39.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 shutdown_connections 2024-04-15T15:25:39.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2c4156e00 0x7fa2c4158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa2a806c980 0x7fa2a806ee40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2c4165ff0 0x7fa2c414c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 --2- 172.21.15.150:0/4248161255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac0051c0 0x7fa2c415a420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 >> 172.21.15.150:0/4248161255 conn(0x7fa2c40b27b0 msgr2=0x7fa2c40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:39.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 shutdown_connections 2024-04-15T15:25:39.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.188+0000 7fa2cc800640 1 -- 172.21.15.150:0/4248161255 wait complete. 2024-04-15T15:25:39.611 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-15T15:25:06.343634+0000', 'modified': '2024-04-15T15:25:37.899562+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5499': {'gid': 5499, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.150:6835/3243584315', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 3243584315}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 3243584315}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5499, 'qdb_cluster': [5499]} 2024-04-15T15:25:39.611 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-15T15:25:39.611 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-15T15:25:39.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.834+0000 7f0028a00640 1 Processor -- start 2024-04-15T15:25:39.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.834+0000 7f0028a00640 1 -- start start 2024-04-15T15:25:39.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.835+0000 7f0028a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00240b0cb0 0x7f00240ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.835+0000 7f0028a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f00240b59b0 0x7f00240b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.835+0000 7f0028a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00240cd940 con 0x7f002415d770 2024-04-15T15:25:39.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.835+0000 7f0028a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f00240ce550 con 0x7f00240b59b0 2024-04-15T15:25:39.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.835+0000 7f0028a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f00240cdc20 con 0x7f00240b0cb0 2024-04-15T15:25:39.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.836+0000 7f0023200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f00240b59b0 0x7f00240b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.836+0000 7f0023200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f00240b59b0 0x7f00240b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36086/0 (socket says 172.21.15.150:36086) 2024-04-15T15:25:39.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.836+0000 7f0023200640 1 -- 172.21.15.150:0/2458772793 learned_addr learned my addr 172.21.15.150:0/2458772793 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:39.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.836+0000 7f0022800640 1 --2- 172.21.15.150:0/2458772793 >> v2:172.21.15.187:3300/0 conn(0x7f00240b0cb0 0x7f00240ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.836+0000 7f0023200640 1 -- 172.21.15.150:0/2458772793 >> v2:172.21.15.187:3300/0 conn(0x7f00240b0cb0 msgr2=0x7f00240ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.836+0000 7f0023200640 1 --2- 172.21.15.150:0/2458772793 >> v2:172.21.15.187:3300/0 conn(0x7f00240b0cb0 0x7f00240ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.836+0000 7f0023200640 1 -- 172.21.15.150:0/2458772793 >> v1:172.21.15.187:6789/0 conn(0x7f002415d770 legacy=0x7f00240cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.836+0000 7f0023200640 1 -- 172.21.15.150:0/2458772793 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00240cdf00 con 0x7f00240b59b0 2024-04-15T15:25:39.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.837+0000 7f0023200640 1 --2- 172.21.15.150:0/2458772793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f00240b5e40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f002406dc10 tx=0x7f001c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d20b20c869132d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:39.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0021400640 1 -- 172.21.15.150:0/2458772793 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c02ca80 con 0x7f00240b59b0 2024-04-15T15:25:39.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0021400640 1 -- 172.21.15.150:0/2458772793 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001c02cc20 con 0x7f00240b59b0 2024-04-15T15:25:39.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0021400640 1 -- 172.21.15.150:0/2458772793 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c0343f0 con 0x7f00240b59b0 2024-04-15T15:25:39.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0028a00640 1 -- 172.21.15.150:0/2458772793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 msgr2=0x7f00240b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0028a00640 1 --2- 172.21.15.150:0/2458772793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f00240b5e40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f002406dc10 tx=0x7f001c02bee0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0028a00640 1 -- 172.21.15.150:0/2458772793 shutdown_connections 2024-04-15T15:25:39.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0028a00640 1 --2- 172.21.15.150:0/2458772793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f00240b5e40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0028a00640 1 --2- 172.21.15.150:0/2458772793 >> v2:172.21.15.187:3300/0 conn(0x7f00240b0cb0 0x7f00240ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.838+0000 7f0028a00640 1 -- 172.21.15.150:0/2458772793 >> 172.21.15.150:0/2458772793 conn(0x7f00240b23a0 msgr2=0x7f00240b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:39.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.839+0000 7f0028a00640 1 -- 172.21.15.150:0/2458772793 shutdown_connections 2024-04-15T15:25:39.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.839+0000 7f0028a00640 1 -- 172.21.15.150:0/2458772793 wait complete. 2024-04-15T15:25:39.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.840+0000 7f0028a00640 1 Processor -- start 2024-04-15T15:25:39.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.840+0000 7f0028a00640 1 -- start start 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.841+0000 7f0028a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00240b0cb0 0x7f0024159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.841+0000 7f0028a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f0024157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.841+0000 7f0028a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002415d770 0x7f0024146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.841+0000 7f0028a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f00240cef20 con 0x7f00240b59b0 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.841+0000 7f0028a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00240b1090 con 0x7f002415d770 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.841+0000 7f0028a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00241576d0 con 0x7f00240b0cb0 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f0024157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f0024157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36090/0 (socket says 172.21.15.150:36090) 2024-04-15T15:25:39.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 -- 172.21.15.150:0/4038712870 learned_addr learned my addr 172.21.15.150:0/4038712870 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0023200640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002415d770 0x7f0024146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0022800640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00240b0cb0 0x7f0024159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 -- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00240b0cb0 msgr2=0x7f0024159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00240b0cb0 0x7f0024159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 -- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002415d770 msgr2=0x7f0024146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002415d770 0x7f0024146fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 -- 172.21.15.150:0/4038712870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0024157900 con 0x7f00240b59b0 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.842+0000 7f0021e00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f0024157190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f001000b3c0 tx=0x7f001000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.843+0000 7f001aa00640 1 -- 172.21.15.150:0/4038712870 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010011830 con 0x7f00240b59b0 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.843+0000 7f001aa00640 1 -- 172.21.15.150:0/4038712870 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0010016070 con 0x7f00240b59b0 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.843+0000 7f0028a00640 1 -- 172.21.15.150:0/4038712870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0024157be0 con 0x7f00240b59b0 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.843+0000 7f001aa00640 1 -- 172.21.15.150:0/4038712870 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010011c50 con 0x7f00240b59b0 2024-04-15T15:25:39.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.843+0000 7f0028a00640 1 -- 172.21.15.150:0/4038712870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00241581a0 con 0x7f00240b59b0 2024-04-15T15:25:39.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.845+0000 7f001aa00640 1 -- 172.21.15.150:0/4038712870 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f001001b460 con 0x7f00240b59b0 2024-04-15T15:25:39.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.846+0000 7f001aa00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f000406c9b0 0x7f000406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.846+0000 7f001aa00640 1 -- 172.21.15.150:0/4038712870 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f001008b760 con 0x7f00240b59b0 2024-04-15T15:25:39.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.847+0000 7f0022800640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f000406c9b0 0x7f000406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.847+0000 7f002c21b740 1 Processor -- start 2024-04-15T15:25:39.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.847+0000 7f002c21b740 1 -- start start 2024-04-15T15:25:39.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f0022800640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f000406c9b0 0x7f000406ee70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f001400b7c0 tx=0x7f0014008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f002c21b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400d7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f002c21b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400b8bc0 0x55e2400914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f002c21b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e240091a00 0x55e24008f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f002c21b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e23fff69c0 con 0x55e2400b87e0 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f002c21b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e23ff9d560 con 0x55e2400b8bc0 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f002c21b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e23fff2980 con 0x55e240091a00 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f0022800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400d7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f0022800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400d7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36094/0 (socket says 172.21.15.150:36094) 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.848+0000 7f0022800640 1 -- 172.21.15.150:0/2475883356 learned_addr learned my addr 172.21.15.150:0/2475883356 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f0023200640 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e240091a00 0x55e24008f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f0021e00640 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400b8bc0 0x55e2400914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f0022800640 1 -- 172.21.15.150:0/2475883356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e240091a00 msgr2=0x55e24008f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f0022800640 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e240091a00 0x55e24008f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f0022800640 1 -- 172.21.15.150:0/2475883356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400b8bc0 msgr2=0x55e2400914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f0022800640 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400b8bc0 0x55e2400914c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f0022800640 1 -- 172.21.15.150:0/2475883356 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e23fd092d0 con 0x55e2400b87e0 2024-04-15T15:25:39.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f0022800640 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400d7920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f001400db60 tx=0x7f0014010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc8a0a29c3247b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:39.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f000f000640 1 -- 172.21.15.150:0/2475883356 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014019050 con 0x55e2400b87e0 2024-04-15T15:25:39.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f000f000640 1 -- 172.21.15.150:0/2475883356 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0014023070 con 0x55e2400b87e0 2024-04-15T15:25:39.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.849+0000 7f000f000640 1 -- 172.21.15.150:0/2475883356 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014012c70 con 0x55e2400b87e0 2024-04-15T15:25:39.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 -- 172.21.15.150:0/2475883356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 msgr2=0x55e2400d7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400d7920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f001400db60 tx=0x7f0014010000 comp rx=0 tx=0).stop 2024-04-15T15:25:39.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 -- 172.21.15.150:0/2475883356 shutdown_connections 2024-04-15T15:25:39.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e240091a00 0x55e24008f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400b8bc0 0x55e2400914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 --2- 172.21.15.150:0/2475883356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400d7920 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 -- 172.21.15.150:0/2475883356 >> 172.21.15.150:0/2475883356 conn(0x55e24008daa0 msgr2=0x55e2400d6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:39.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 -- 172.21.15.150:0/2475883356 shutdown_connections 2024-04-15T15:25:39.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.850+0000 7f002c21b740 1 -- 172.21.15.150:0/2475883356 wait complete. 2024-04-15T15:25:39.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.851+0000 7f002c21b740 1 -- start start 2024-04-15T15:25:39.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.851+0000 7f002c21b740 1 Processor -- start 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f002c21b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 0x55e2400da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f002c21b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400e6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f002c21b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f002c21b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e24008b8d0 con 0x55e2400b8bc0 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f002c21b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e23ffb0920 con 0x55e2400d8750 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f002c21b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e240091e80 con 0x55e2400b87e0 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f002c21b740 10 client.0 _ll_register_callbacks cb 0x55e23ffb0310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f0021e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f0021e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400e6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36096/0 (socket says 172.21.15.150:36096) 2024-04-15T15:25:39.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.852+0000 7f0021e00640 1 -- 172.21.15.150:0/3470422943 learned_addr learned my addr 172.21.15.150:0/3470422943 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0022800640 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 0x55e2400da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0023200640 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0021e00640 1 -- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 msgr2=0x55e2400da470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0021e00640 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 0x55e2400da470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0021e00640 1 -- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 msgr2=0x55e2400d8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0021e00640 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0021e00640 1 -- 172.21.15.150:0/3470422943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e23fdd36b0 con 0x55e2400b8bc0 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0023200640 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0021e00640 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400e6640 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0010089af0 tx=0x7f0010059480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43271a21ebadfda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0003400640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0003400640 1 -- 172.21.15.150:0/3470422943 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010057700 con 0x55e2400b8bc0 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f0003400640 1 -- 172.21.15.150:0/3470422943 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f0010061a90 con 0x55e2400b8bc0 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.853+0000 7f002c21b740 1 -- 172.21.15.150:0/3470422943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0014023070 con 0x55e2400b8bc0 2024-04-15T15:25:39.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f0003400640 1 -- 172.21.15.150:0/3470422943 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0010061c90 con 0x55e2400b8bc0 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f002c21b740 10 client.5532 fetch_fsmap learned FSMap version 22 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f002c21b740 1 -- 172.21.15.150:0/3470422943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55e2400d9990 con 0x55e2400b8bc0 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f0003400640 1 -- 172.21.15.150:0/3470422943 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0014023070 con 0x55e2400b8bc0 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f0003400640 1 -- 172.21.15.150:0/3470422943 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f0010058790 con 0x55e2400b8bc0 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f002c21b740 10 client.5532 fetch_fsmap finished waiting for FSMap version 22 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f002c21b740 10 client.5532 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f002c21b740 20 client.5532 populate_metadata read hostname 'smithi150' 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f002c21b740 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] conn(0x55e2400f2be0 0x55e2400f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f002c21b740 4 client.5532 mds_command: new command op to 5499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.854+0000 7f002c21b740 1 -- 172.21.15.150:0/3470422943 --> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e23ff9cb70 con 0x55e2400f2be0 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.855+0000 7f0022800640 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] conn(0x55e2400f2be0 0x55e2400f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.855+0000 7f0022800640 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] conn(0x55e2400f2be0 0x55e2400f4fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b5405d445689a7 server_cookie=77531d9b862ee417 in_seq=0 out_seq=0 2024-04-15T15:25:39.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.855+0000 7f0003400640 10 client.5532 ms_handle_connect on v2:172.21.15.150:6833/3243584315 2024-04-15T15:25:39.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f0003400640 1 -- 172.21.15.150:0/3470422943 <== mds.0 v2:172.21.15.150:6833/3243584315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e23ff9cb70 con 0x55e2400f2be0 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f0003400640 10 client.5532 handle_command_reply: tid=0 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 client.5532 shutdown 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 -- 172.21.15.150:0/3470422943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 msgr2=0x55e2400e6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400e6640 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0010089af0 tx=0x7f0010059480 comp rx=0 tx=0).stop 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 -- 172.21.15.150:0/3470422943 shutdown_connections 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] conn(0x55e2400f2be0 0x55e2400f4fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400e6640 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f0003400640 0 client.5532 ms_handle_reset on v2:172.21.15.150:6833/3243584315 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 --2- 172.21.15.150:0/3470422943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 0x55e2400da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.856+0000 7f002c21b740 1 -- 172.21.15.150:0/3470422943 >> 172.21.15.150:0/3470422943 conn(0x55e240091a00 msgr2=0x55e2400d65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.857+0000 7f002c21b740 1 -- 172.21.15.150:0/3470422943 shutdown_connections 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.857+0000 7f002c21b740 1 -- 172.21.15.150:0/3470422943 wait complete. 2024-04-15T15:25:39.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.857+0000 7f002c21b740 20 client.5532 trim_cache size 0 max 16384 2024-04-15T15:25:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.876+0000 7f002c21b740 1 Processor -- start 2024-04-15T15:25:39.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.876+0000 7f002c21b740 1 -- start start 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.877+0000 7f002c21b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 0x55e2400aac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.877+0000 7f002c21b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400d65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.877+0000 7f002c21b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.877+0000 7f002c21b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f00241423f0 con 0x55e2400b8bc0 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.877+0000 7f002c21b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0024141750 con 0x55e2400d8750 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.877+0000 7f002c21b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0024141da0 con 0x55e2400b87e0 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.877+0000 7f0021e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400d65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0021e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400d65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36100/0 (socket says 172.21.15.150:36100) 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0021e00640 1 -- 172.21.15.150:0/1426939749 learned_addr learned my addr 172.21.15.150:0/1426939749 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0023200640 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0021e00640 1 -- 172.21.15.150:0/1426939749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 msgr2=0x55e2400aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0022800640 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 0x55e2400aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0021e00640 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 0x55e2400aac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0021e00640 1 -- 172.21.15.150:0/1426939749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 msgr2=0x55e2400d6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0021e00640 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0021e00640 1 -- 172.21.15.150:0/1426939749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2400ad7d0 con 0x55e2400b8bc0 2024-04-15T15:25:39.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.878+0000 7f0021e00640 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400d65b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f001000bd80 tx=0x7f001001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9057cd0c2750c912 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:39.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f000e600640 1 -- 172.21.15.150:0/1426939749 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010057700 con 0x55e2400b8bc0 2024-04-15T15:25:39.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f000e600640 1 -- 172.21.15.150:0/1426939749 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0010058790 con 0x55e2400b8bc0 2024-04-15T15:25:39.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f000e600640 1 -- 172.21.15.150:0/1426939749 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010071b70 con 0x55e2400b8bc0 2024-04-15T15:25:39.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 -- 172.21.15.150:0/1426939749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 msgr2=0x55e2400d65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400d65b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f001000bd80 tx=0x7f001001d000 comp rx=0 tx=0).stop 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 -- 172.21.15.150:0/1426939749 shutdown_connections 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e2400d6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b8bc0 0x55e2400d65b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 --2- 172.21.15.150:0/1426939749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b87e0 0x55e2400aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 -- 172.21.15.150:0/1426939749 >> 172.21.15.150:0/1426939749 conn(0x55e240091a00 msgr2=0x55e2400e6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 -- 172.21.15.150:0/1426939749 shutdown_connections 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.879+0000 7f002c21b740 1 -- 172.21.15.150:0/1426939749 wait complete. 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.880+0000 7f002c21b740 1 -- start start 2024-04-15T15:25:39.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.880+0000 7f002c21b740 1 Processor -- start 2024-04-15T15:25:39.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.881+0000 7f002c21b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400da600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.881+0000 7f002c21b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b8bc0 0x55e2401516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.881+0000 7f002c21b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e240090730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.881+0000 7f002c21b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e2400ac000 con 0x55e2400b87e0 2024-04-15T15:25:39.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.881+0000 7f002c21b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e2400d9cc0 con 0x55e2400d8750 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.881+0000 7f002c21b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e2400ac650 con 0x55e2400b8bc0 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.881+0000 7f002c21b740 10 client.0 _ll_register_callbacks cb 0x7f0010059a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400da600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400da600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36108/0 (socket says 172.21.15.150:36108) 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 -- 172.21.15.150:0/3070992776 learned_addr learned my addr 172.21.15.150:0/3070992776 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0021e00640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b8bc0 0x55e2401516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0023200640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e240090730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 -- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b8bc0 msgr2=0x55e2401516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b8bc0 0x55e2401516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 -- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 msgr2=0x55e240090730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e240090730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 -- 172.21.15.150:0/3070992776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e240151c10 con 0x55e2400b87e0 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0021e00640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b8bc0 0x55e2401516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0023200640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e240090730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:39.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0022800640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400da600 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f001400db30 tx=0x7f0014004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13a2a6ade887b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0003400640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.882+0000 7f0003400640 1 -- 172.21.15.150:0/3070992776 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014038060 con 0x55e2400b87e0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f0003400640 1 -- 172.21.15.150:0/3070992776 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f0014032b50 con 0x55e2400b87e0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 1 -- 172.21.15.150:0/3070992776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0010058790 con 0x55e2400b87e0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f0003400640 1 -- 172.21.15.150:0/3070992776 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f001403e050 con 0x55e2400b87e0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 10 client.5538 fetch_fsmap learned FSMap version 22 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 1 -- 172.21.15.150:0/3070992776 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55e2400d6b40 con 0x55e2400b87e0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f0003400640 1 -- 172.21.15.150:0/3070992776 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0010058790 con 0x55e2400b87e0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f0003400640 1 -- 172.21.15.150:0/3070992776 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f0014041020 con 0x55e2400b87e0 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 10 client.5538 fetch_fsmap finished waiting for FSMap version 22 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 10 client.5538 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 20 client.5538 populate_metadata read hostname 'smithi150' 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] conn(0x55e24008f390 0x55e24008f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 4 client.5538 mds_command: new command op to 5499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-15T15:25:39.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.883+0000 7f002c21b740 1 -- 172.21.15.150:0/3070992776 --> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] -- command(tid 0: {"prefix": "status"}) -- 0x55e2400043e0 con 0x55e24008f390 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.884+0000 7f0021e00640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] conn(0x55e24008f390 0x55e24008f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.884+0000 7f0021e00640 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] conn(0x55e24008f390 0x55e24008f770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b0a6bd8e634fe2e server_cookie=8375d3174b161c09 in_seq=0 out_seq=0 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.884+0000 7f0003400640 10 client.5538 ms_handle_connect on v2:172.21.15.150:6833/3243584315 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.884+0000 7f0003400640 1 -- 172.21.15.150:0/3070992776 <== mds.0 v2:172.21.15.150:6833/3243584315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e2400043e0 con 0x55e24008f390 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.884+0000 7f0003400640 10 client.5538 handle_command_reply: tid=0 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.884+0000 7f002c21b740 1 client.5538 shutdown 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.884+0000 7f002c21b740 1 -- 172.21.15.150:0/3070992776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 msgr2=0x55e2400da600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.884+0000 7f002c21b740 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400da600 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f001400db30 tx=0x7f0014004730 comp rx=0 tx=0).stop 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 1 -- 172.21.15.150:0/3070992776 shutdown_connections 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.150:6833/3243584315,v1:172.21.15.150:6835/3243584315] conn(0x55e24008f390 0x55e24008f770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2400d8750 0x55e240090730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e2400b8bc0 0x55e2401516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 1 --2- 172.21.15.150:0/3070992776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2400b87e0 0x55e2400da600 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f0003400640 0 client.5538 ms_handle_reset on v2:172.21.15.150:6833/3243584315 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 1 -- 172.21.15.150:0/3070992776 >> 172.21.15.150:0/3070992776 conn(0x55e240091a00 msgr2=0x55e2400ab2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 1 -- 172.21.15.150:0/3070992776 shutdown_connections 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 1 -- 172.21.15.150:0/3070992776 wait complete. 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f002c21b740 20 client.5538 trim_cache size 0 max 16384 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4", 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 5499, 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-04-15T15:25:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-04-15T15:25:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-04-15T15:25:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 4.9996592529999999, 2024-04-15T15:25:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 22, 2024-04-15T15:25:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 36, 2024-04-15T15:25:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 36, 2024-04-15T15:25:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 5.8098242979999997 2024-04-15T15:25:39.892 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f0028a00640 1 -- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f000406c9b0 msgr2=0x7f000406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f0028a00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f000406c9b0 0x7f000406ee70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f001400b7c0 tx=0x7f0014008040 comp rx=0 tx=0).stop 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.885+0000 7f0028a00640 1 -- 172.21.15.150:0/4038712870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 msgr2=0x7f0024157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f0024157190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f001000b3c0 tx=0x7f001000b880 comp rx=0 tx=0).stop 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 -- 172.21.15.150:0/4038712870 shutdown_connections 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f002415d770 0x7f0024146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00240b59b0 0x7f0024157190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f000406c9b0 0x7f000406ee70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 --2- 172.21.15.150:0/4038712870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00240b0cb0 0x7f0024159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 -- 172.21.15.150:0/4038712870 >> 172.21.15.150:0/4038712870 conn(0x7f00240b23a0 msgr2=0x7f002415bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:39.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 -- 172.21.15.150:0/4038712870 shutdown_connections 2024-04-15T15:25:39.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:39.886+0000 7f0028a00640 1 -- 172.21.15.150:0/4038712870 wait complete. 2024-04-15T15:25:39.897 INFO:tasks.ceph.mds.a.smithi150.stderr:2024-04-15T15:25:39.895+0000 7f03e2200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:25:39.904 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-15T15:25:39.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:25:40.017 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-15T15:25:40.141 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-15T15:25:40.142 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:25:40.142 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:25:40.142 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:40.199 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:25:40.200 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:40.253 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:40.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.467+0000 7f0595000640 1 Processor -- start 2024-04-15T15:25:40.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.467+0000 7f0595000640 1 -- start start 2024-04-15T15:25:40.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.468+0000 7f0595000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f05901461f0 0x7f05900ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.468+0000 7f0595000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05900cedf0 0x7f05900b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.468+0000 7f0595000640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05900b4be0 con 0x7f05900b3e10 2024-04-15T15:25:40.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.468+0000 7f0595000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f05900a9640 con 0x7f05901461f0 2024-04-15T15:25:40.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.468+0000 7f0595000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f05900cf250 con 0x7f05900cedf0 2024-04-15T15:25:40.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.469+0000 7f058e800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f05900b3e10 0x7f05900cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47686/0 (socket says 172.21.15.150:47686) 2024-04-15T15:25:40.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.469+0000 7f058e800640 1 -- 172.21.15.150:0/2961042553 learned_addr learned my addr 172.21.15.150:0/2961042553 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.469+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274132867 0 0) 0x7f05900b4be0 con 0x7f05900b3e10 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.469+0000 7f058d400640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05900cedf0 0x7f05900b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.469+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0574003520 con 0x7f05900b3e10 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.469+0000 7f058de00640 1 --2- 172.21.15.150:0/2961042553 >> v2:172.21.15.187:3300/0 conn(0x7f05901461f0 0x7f05900ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2759726708 0 0) 0x7f0574003520 con 0x7f05900b3e10 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05900cedf0 msgr2=0x7f05900b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05900cedf0 0x7f05900b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 >> v2:172.21.15.187:3300/0 conn(0x7f05901461f0 msgr2=0x7f05900ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 --2- 172.21.15.150:0/2961042553 >> v2:172.21.15.187:3300/0 conn(0x7f05901461f0 0x7f05900ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05900b4620 con 0x7f05900b3e10 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f0588001670 con 0x7f05900b3e10 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 >> v1:172.21.15.150:6789/0 conn(0x7f05900b3e10 legacy=0x7f05900cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05900cedf0 0x7f05740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0574003f30 0x7f0574004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f057400b9c0 con 0x7f0574004d40 2024-04-15T15:25:40.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f057400bb40 con 0x7f0574003f30 2024-04-15T15:25:40.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f057400bd10 con 0x7f05900cedf0 2024-04-15T15:25:40.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.470+0000 7f058e800640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.471+0000 7f058e800640 1 -- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05900cedf0 msgr2=0x7f05740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:40.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.471+0000 7f058e800640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05900cedf0 0x7f05740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.471+0000 7f058e800640 1 -- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0574003f30 msgr2=0x7f0574004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:40.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.471+0000 7f058e800640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0574003f30 0x7f0574004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.471+0000 7f058e800640 1 -- 172.21.15.150:0/2961042553 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0574003520 con 0x7f0574004d40 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.472+0000 7f058e800640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05740051c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f058800c240 tx=0x7f0574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4935027149be6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.472+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588007d10 con 0x7f0574004d40 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.472+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0588030070 con 0x7f0574004d40 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.472+0000 7f058ca00640 1 -- 172.21.15.150:0/2961042553 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058802cda0 con 0x7f0574004d40 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.472+0000 7f0595000640 1 -- 172.21.15.150:0/2961042553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 msgr2=0x7f05740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.472+0000 7f0595000640 1 --2- 172.21.15.150:0/2961042553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05740051c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f058800c240 tx=0x7f0574005830 comp rx=0 tx=0).stop 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.472+0000 7f058e800640 1 -- 172.21.15.150:0/2961042553 reap_dead start 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.473+0000 7f0595000640 1 -- 172.21.15.150:0/2961042553 shutdown_connections 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.473+0000 7f0595000640 1 -- 172.21.15.150:0/2961042553 >> 172.21.15.150:0/2961042553 conn(0x7f05900b27b0 msgr2=0x7f05900b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.473+0000 7f0595000640 1 -- 172.21.15.150:0/2961042553 shutdown_connections 2024-04-15T15:25:40.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.473+0000 7f0595000640 1 -- 172.21.15.150:0/2961042553 wait complete. 2024-04-15T15:25:40.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.474+0000 7f0595000640 1 Processor -- start 2024-04-15T15:25:40.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.475+0000 7f0595000640 1 -- start start 2024-04-15T15:25:40.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.475+0000 7f0595000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05901376a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f0595000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0590137be0 0x7f05901f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f0595000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05901f45a0 0x7f05901358b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f0595000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f059015d290 con 0x7f0574004d40 2024-04-15T15:25:40.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f0595000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f059015cfb0 con 0x7f05901f45a0 2024-04-15T15:25:40.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f0595000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f05901465d0 con 0x7f0590137be0 2024-04-15T15:25:40.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f058de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05901376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f058de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05901376a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36116/0 (socket says 172.21.15.150:36116) 2024-04-15T15:25:40.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f058de00640 1 -- 172.21.15.150:0/580566769 learned_addr learned my addr 172.21.15.150:0/580566769 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:40.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f058de00640 1 -- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0590137be0 msgr2=0x7f05901f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f058de00640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0590137be0 0x7f05901f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f058de00640 1 -- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05901f45a0 msgr2=0x7f05901358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f058de00640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05901f45a0 0x7f05901358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.476+0000 7f058de00640 1 -- 172.21.15.150:0/580566769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0590136600 con 0x7f0574004d40 2024-04-15T15:25:40.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.477+0000 7f058de00640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05901376a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0580007e10 tx=0x7f058000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:40.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.477+0000 7f0586000640 1 -- 172.21.15.150:0/580566769 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580014620 con 0x7f0574004d40 2024-04-15T15:25:40.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.477+0000 7f0586000640 1 -- 172.21.15.150:0/580566769 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0580019070 con 0x7f0574004d40 2024-04-15T15:25:40.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.477+0000 7f0586000640 1 -- 172.21.15.150:0/580566769 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580014a20 con 0x7f0574004d40 2024-04-15T15:25:40.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.477+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05901368e0 con 0x7f0574004d40 2024-04-15T15:25:40.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.477+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0590136ea0 con 0x7f0574004d40 2024-04-15T15:25:40.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.478+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0590068c80 con 0x7f0574004d40 2024-04-15T15:25:40.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.479+0000 7f0586000640 1 -- 172.21.15.150:0/580566769 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0580014bc0 con 0x7f0574004d40 2024-04-15T15:25:40.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.480+0000 7f0586000640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f057006c980 0x7f057006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.480+0000 7f0586000640 1 -- 172.21.15.150:0/580566769 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f058008c560 con 0x7f0574004d40 2024-04-15T15:25:40.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.481+0000 7f058d400640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f057006c980 0x7f057006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.482+0000 7f058d400640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f057006c980 0x7f057006ee40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0590143ac0 tx=0x7f057c0094e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:40.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.485+0000 7f0586000640 1 -- 172.21.15.150:0/580566769 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f058005a840 con 0x7f0574004d40 2024-04-15T15:25:40.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.686+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05900700e0 con 0x7f0574004d40 2024-04-15T15:25:40.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.689+0000 7f0586000640 1 -- 172.21.15.150:0/580566769 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16848 (secure 0 0 0) 0x7f058005a140 con 0x7f0574004d40 2024-04-15T15:25:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:40.691 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":36,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:33.990388+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/3327940880":"2024-04-16T15:25:33.990360+0000","172.21.15.150:6833/3327940880":"2024-04-16T15:25:33.990360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f057006c980 msgr2=0x7f057006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f057006c980 0x7f057006ee40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0590143ac0 tx=0x7f057c0094e0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 msgr2=0x7f05901376a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05901376a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0580007e10 tx=0x7f058000e4b0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 shutdown_connections 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05901f45a0 0x7f05901358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f057006c980 0x7f057006ee40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0590137be0 0x7f05901f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 --2- 172.21.15.150:0/580566769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0574004d40 0x7f05901376a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 >> 172.21.15.150:0/580566769 conn(0x7f05900b27b0 msgr2=0x7f0590145190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:40.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 shutdown_connections 2024-04-15T15:25:40.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.692+0000 7f0595000640 1 -- 172.21.15.150:0/580566769 wait complete. 2024-04-15T15:25:40.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:25:04.190900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:25:05.324314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:25:40.714 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:25:40.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.928+0000 7fcbe8c00640 1 Processor -- start 2024-04-15T15:25:40.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.929+0000 7fcbe8c00640 1 -- start start 2024-04-15T15:25:40.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.930+0000 7fcbe8c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcbe406fc80 0x7fcbe415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.930+0000 7fcbe8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe4072150 0x7fcbe40725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.930+0000 7fcbe8c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe415d520 con 0x7fcbe415a440 2024-04-15T15:25:40.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.930+0000 7fcbe8c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe406e280 con 0x7fcbe406fc80 2024-04-15T15:25:40.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.930+0000 7fcbe8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe415d800 con 0x7fcbe4072150 2024-04-15T15:25:40.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.930+0000 7fcbe2800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe4072150 0x7fcbe40725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.931+0000 7fcbdbe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcbe406fc80 0x7fcbe415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.931+0000 7fcbe2800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe4072150 0x7fcbe40725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:39106/0 (socket says 172.21.15.150:39106) 2024-04-15T15:25:40.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.931+0000 7fcbe2800640 1 -- 172.21.15.150:0/3446688242 learned_addr learned my addr 172.21.15.150:0/3446688242 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:40.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.931+0000 7fcbe1e00640 1 --1- 172.21.15.150:0/3446688242 >> v1:172.21.15.187:6789/0 conn(0x7fcbe415a440 0x7fcbe40cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:44592/0 (socket says 172.21.15.150:44592) 2024-04-15T15:25:40.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.931+0000 7fcbe2800640 1 -- 172.21.15.150:0/3446688242 >> v2:172.21.15.187:3300/0 conn(0x7fcbe406fc80 msgr2=0x7fcbe415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:40.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.931+0000 7fcbe2800640 1 --2- 172.21.15.150:0/3446688242 >> v2:172.21.15.187:3300/0 conn(0x7fcbe406fc80 0x7fcbe415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.931+0000 7fcbe2800640 1 -- 172.21.15.150:0/3446688242 >> v1:172.21.15.187:6789/0 conn(0x7fcbe415a440 legacy=0x7fcbe40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.931+0000 7fcbe2800640 1 -- 172.21.15.150:0/3446688242 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe415dae0 con 0x7fcbe4072150 2024-04-15T15:25:40.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.933+0000 7fcbe2800640 1 --2- 172.21.15.150:0/3446688242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe4072150 0x7fcbe40725e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009e50 tx=0x7fcbdc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e22a3128f75deafa server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:40.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.933+0000 7fcbe1400640 1 -- 172.21.15.150:0/3446688242 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc02cb00 con 0x7fcbe4072150 2024-04-15T15:25:40.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.933+0000 7fcbe1400640 1 -- 172.21.15.150:0/3446688242 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbdc02cca0 con 0x7fcbe4072150 2024-04-15T15:25:40.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.933+0000 7fcbe1400640 1 -- 172.21.15.150:0/3446688242 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc034610 con 0x7fcbe4072150 2024-04-15T15:25:40.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.934+0000 7fcbe8c00640 1 -- 172.21.15.150:0/3446688242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe4072150 msgr2=0x7fcbe40725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.934+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/3446688242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe4072150 0x7fcbe40725e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009e50 tx=0x7fcbdc02bf60 comp rx=0 tx=0).stop 2024-04-15T15:25:40.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.934+0000 7fcbe8c00640 1 -- 172.21.15.150:0/3446688242 shutdown_connections 2024-04-15T15:25:40.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.934+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/3446688242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe4072150 0x7fcbe40725e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.934+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/3446688242 >> v2:172.21.15.187:3300/0 conn(0x7fcbe406fc80 0x7fcbe415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.934+0000 7fcbe8c00640 1 -- 172.21.15.150:0/3446688242 >> 172.21.15.150:0/3446688242 conn(0x7fcbe40b2460 msgr2=0x7fcbe40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:40.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.934+0000 7fcbe8c00640 1 -- 172.21.15.150:0/3446688242 shutdown_connections 2024-04-15T15:25:40.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.934+0000 7fcbe8c00640 1 -- 172.21.15.150:0/3446688242 wait complete. 2024-04-15T15:25:40.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.936+0000 7fcbe8c00640 1 Processor -- start 2024-04-15T15:25:40.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.936+0000 7fcbe8c00640 1 -- start start 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe8c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe406fc80 0x7fcbe40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe8c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe4072150 0x7fcbe4144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe415a440 0x7fcbe413bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe8c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe415ddc0 con 0x7fcbe406fc80 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe8c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe415a820 con 0x7fcbe4072150 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe406f510 con 0x7fcbe415a440 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe406fc80 0x7fcbe40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe406fc80 0x7fcbe40b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36126/0 (socket says 172.21.15.150:36126) 2024-04-15T15:25:40.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.937+0000 7fcbe1e00640 1 -- 172.21.15.150:0/720323330 learned_addr learned my addr 172.21.15.150:0/720323330 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:40.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbe1e00640 1 -- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe415a440 msgr2=0x7fcbe413bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbe1e00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe415a440 0x7fcbe413bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbe1e00640 1 -- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe4072150 msgr2=0x7fcbe4144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:40.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbe1e00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe4072150 0x7fcbe4144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:40.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbe1e00640 1 -- 172.21.15.150:0/720323330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe413d500 con 0x7fcbe406fc80 2024-04-15T15:25:40.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbe1e00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe406fc80 0x7fcbe40b3ac0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c430 tx=0x7fcbd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:40.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbda000640 1 -- 172.21.15.150:0/720323330 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40074e0 con 0x7fcbe406fc80 2024-04-15T15:25:40.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbda000640 1 -- 172.21.15.150:0/720323330 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd4007680 con 0x7fcbe406fc80 2024-04-15T15:25:40.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbda000640 1 -- 172.21.15.150:0/720323330 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4004a00 con 0x7fcbe406fc80 2024-04-15T15:25:40.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe413d7e0 con 0x7fcbe406fc80 2024-04-15T15:25:40.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.938+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe4142dd0 con 0x7fcbe406fc80 2024-04-15T15:25:40.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.939+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe4068c80 con 0x7fcbe406fc80 2024-04-15T15:25:40.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.940+0000 7fcbda000640 1 -- 172.21.15.150:0/720323330 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcbd401a020 con 0x7fcbe406fc80 2024-04-15T15:25:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.941+0000 7fcbda000640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcbc006c980 0x7fcbc006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.941+0000 7fcbda000640 1 -- 172.21.15.150:0/720323330 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fcbd408b4b0 con 0x7fcbe406fc80 2024-04-15T15:25:40.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.942+0000 7fcbdbe00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcbc006c980 0x7fcbc006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:40.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.943+0000 7fcbdbe00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcbc006c980 0x7fcbc006ee40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc006e80 tx=0x7fcbcc006dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:40.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:40.946+0000 7fcbda000640 1 -- 172.21.15.150:0/720323330 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbd4010300 con 0x7fcbe406fc80 2024-04-15T15:25:41.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.148+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcbe40afdd0 con 0x7fcbe406fc80 2024-04-15T15:25:41.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.149+0000 7fcbda000640 1 -- 172.21.15.150:0/720323330 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7fcbd40597d0 con 0x7fcbe406fc80 2024-04-15T15:25:41.150 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:25:41.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.151+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcbc006c980 msgr2=0x7fcbc006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.151+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcbc006c980 0x7fcbc006ee40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc006e80 tx=0x7fcbcc006dd0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.151+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe406fc80 msgr2=0x7fcbe40b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.151+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe406fc80 0x7fcbe40b3ac0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c430 tx=0x7fcbd400c8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.152+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 shutdown_connections 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.152+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcbc006c980 0x7fcbc006ee40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.152+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe415a440 0x7fcbe413bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.152+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe4072150 0x7fcbe4144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.152+0000 7fcbe8c00640 1 --2- 172.21.15.150:0/720323330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe406fc80 0x7fcbe40b3ac0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.152+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 >> 172.21.15.150:0/720323330 conn(0x7fcbe40b2460 msgr2=0x7fcbe415ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.152+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 shutdown_connections 2024-04-15T15:25:41.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.152+0000 7fcbe8c00640 1 -- 172.21.15.150:0/720323330 wait complete. 2024-04-15T15:25:41.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:25:41.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.380+0000 7f8648400640 1 Processor -- start 2024-04-15T15:25:41.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.380+0000 7f8648400640 1 -- start start 2024-04-15T15:25:41.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8648400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 0x7f864015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8648400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f864006fc80 0x7f86400cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8648400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f864015d520 con 0x7f86400cd620 2024-04-15T15:25:41.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8648400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f864006e280 con 0x7f864006fc80 2024-04-15T15:25:41.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8648400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f864015d800 con 0x7f864015a440 2024-04-15T15:25:41.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8646400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f86400cd620 0x7f8640072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47692/0 (socket says 172.21.15.150:47692) 2024-04-15T15:25:41.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8646400640 1 -- 172.21.15.150:0/859068153 learned_addr learned my addr 172.21.15.150:0/859068153 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:41.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8645a00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 0x7f864015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.382+0000 7f8645000640 1 --2- 172.21.15.150:0/859068153 >> v2:172.21.15.187:3300/0 conn(0x7f864006fc80 0x7f86400cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4074650042 0 0) 0x7f864015d520 con 0x7f86400cd620 2024-04-15T15:25:41.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8628003520 con 0x7f86400cd620 2024-04-15T15:25:41.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3561768791 0 0) 0x7f8628003520 con 0x7f86400cd620 2024-04-15T15:25:41.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 msgr2=0x7f864015cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:41.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 0x7f864015cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 >> v2:172.21.15.187:3300/0 conn(0x7f864006fc80 msgr2=0x7f86400cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 --2- 172.21.15.150:0/859068153 >> v2:172.21.15.187:3300/0 conn(0x7f864006fc80 0x7f86400cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864015dae0 con 0x7f86400cd620 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f86380019f0 con 0x7f86400cd620 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 >> v1:172.21.15.150:6789/0 conn(0x7f86400cd620 legacy=0x7f8640072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8628003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8628004800 0x7f8628004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 0x7f86280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f864006e280 con 0x7f86280039b0 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f864015d800 con 0x7f8628004800 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.383+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f862800b9c0 con 0x7f864015a440 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8645a00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8628003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8646400640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 0x7f86280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8645000640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8628004800 0x7f8628004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8645a00640 1 -- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 msgr2=0x7f86280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:41.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8645a00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 0x7f86280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8645a00640 1 -- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8628004800 msgr2=0x7f8628004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:41.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8645a00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8628004800 0x7f8628004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8645a00640 1 -- 172.21.15.150:0/859068153 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8628003520 con 0x7f86280039b0 2024-04-15T15:25:41.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8646400640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f864015a440 0x7f86280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:41.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.384+0000 7f8645000640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8628004800 0x7f8628004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:41.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.385+0000 7f8645a00640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8628003dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8634009ac0 tx=0x7f8628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cbf670b614def4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:41.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.385+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634039070 con 0x7f86280039b0 2024-04-15T15:25:41.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f863402c6b0 con 0x7f86280039b0 2024-04-15T15:25:41.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f863fe00640 1 -- 172.21.15.150:0/859068153 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634009690 con 0x7f86280039b0 2024-04-15T15:25:41.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f8648400640 1 -- 172.21.15.150:0/859068153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 msgr2=0x7f8628003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f8648400640 1 --2- 172.21.15.150:0/859068153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8628003dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8634009ac0 tx=0x7f8628005830 comp rx=0 tx=0).stop 2024-04-15T15:25:41.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f8646400640 1 -- 172.21.15.150:0/859068153 reap_dead start 2024-04-15T15:25:41.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f8648400640 1 -- 172.21.15.150:0/859068153 shutdown_connections 2024-04-15T15:25:41.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f8648400640 1 -- 172.21.15.150:0/859068153 >> 172.21.15.150:0/859068153 conn(0x7f86400b2460 msgr2=0x7f86400b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:41.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f8648400640 1 -- 172.21.15.150:0/859068153 shutdown_connections 2024-04-15T15:25:41.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.386+0000 7f8648400640 1 -- 172.21.15.150:0/859068153 wait complete. 2024-04-15T15:25:41.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.388+0000 7f8648400640 1 Processor -- start 2024-04-15T15:25:41.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.388+0000 7f8648400640 1 -- start start 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.389+0000 7f8648400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8640142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.389+0000 7f8648400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86400b3520 0x7f86400b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.389+0000 7f8648400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86401f0630 0x7f86401f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.389+0000 7f8648400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f864006f510 con 0x7f86280039b0 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.389+0000 7f8648400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f864015ddc0 con 0x7f86400b3520 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.389+0000 7f8648400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8640072860 con 0x7f86401f0630 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.389+0000 7f8645a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8640142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8640142890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36146/0 (socket says 172.21.15.150:36146) 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645a00640 1 -- 172.21.15.150:0/1529411905 learned_addr learned my addr 172.21.15.150:0/1529411905 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645000640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86400b3520 0x7f86400b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8646400640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86401f0630 0x7f86401f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645a00640 1 -- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86401f0630 msgr2=0x7f86401f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645a00640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86401f0630 0x7f86401f0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645a00640 1 -- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86400b3520 msgr2=0x7f86400b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645a00640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86400b3520 0x7f86400b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645a00640 1 -- 172.21.15.150:0/1529411905 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86400b3fc0 con 0x7f86280039b0 2024-04-15T15:25:41.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8646400640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86401f0630 0x7f86401f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:41.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.390+0000 7f8645a00640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8640142890 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8634009be0 tx=0x7f8634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:41.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.391+0000 7f863d600640 1 -- 172.21.15.150:0/1529411905 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634042020 con 0x7f86280039b0 2024-04-15T15:25:41.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.391+0000 7f863d600640 1 -- 172.21.15.150:0/1529411905 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f863403a040 con 0x7f86280039b0 2024-04-15T15:25:41.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.391+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86400b42a0 con 0x7f86280039b0 2024-04-15T15:25:41.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.391+0000 7f863d600640 1 -- 172.21.15.150:0/1529411905 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634039070 con 0x7f86280039b0 2024-04-15T15:25:41.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.391+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86400b4860 con 0x7f86280039b0 2024-04-15T15:25:41.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.392+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8640068c80 con 0x7f86280039b0 2024-04-15T15:25:41.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.393+0000 7f863d600640 1 -- 172.21.15.150:0/1529411905 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8634045020 con 0x7f86280039b0 2024-04-15T15:25:41.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.394+0000 7f863d600640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f862006c950 0x7f862006ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.394+0000 7f863d600640 1 -- 172.21.15.150:0/1529411905 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f86340ad490 con 0x7f86280039b0 2024-04-15T15:25:41.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.394+0000 7f8645000640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f862006c950 0x7f862006ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.396+0000 7f8645000640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f862006c950 0x7f862006ee10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86400b1ba0 tx=0x7f862c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:41.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.398+0000 7f863d600640 1 -- 172.21.15.150:0/1529411905 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f863404c030 con 0x7f86280039b0 2024-04-15T15:25:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.596+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f86400cdd60 con 0x7f86280039b0 2024-04-15T15:25:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.596+0000 7f863d600640 1 -- 172.21.15.150:0/1529411905 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f863407b770 con 0x7f86280039b0 2024-04-15T15:25:41.598 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f862006c950 msgr2=0x7f862006ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f862006c950 0x7f862006ee10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86400b1ba0 tx=0x7f862c008040 comp rx=0 tx=0).stop 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 msgr2=0x7f8640142890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8640142890 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8634009be0 tx=0x7f8634004730 comp rx=0 tx=0).stop 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 shutdown_connections 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f862006c950 0x7f862006ee10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86401f0630 0x7f86401f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86400b3520 0x7f86400b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 --2- 172.21.15.150:0/1529411905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86280039b0 0x7f8640142890 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 >> 172.21.15.150:0/1529411905 conn(0x7f86400b2460 msgr2=0x7f8640144bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 shutdown_connections 2024-04-15T15:25:41.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.599+0000 7f8648400640 1 -- 172.21.15.150:0/1529411905 wait complete. 2024-04-15T15:25:41.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:25:41.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.834+0000 7f7ef8e00640 1 Processor -- start 2024-04-15T15:25:41.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.834+0000 7f7ef8e00640 1 -- start start 2024-04-15T15:25:41.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.836+0000 7f7ef8e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7ef415c680 0x7f7ef4162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.836+0000 7f7ef8e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef40d2b50 0x7f7ef40d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.836+0000 7f7ef8e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef40b70d0 con 0x7f7ef40b6c70 2024-04-15T15:25:41.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.836+0000 7f7ef8e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef4162dd0 con 0x7f7ef415c680 2024-04-15T15:25:41.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.836+0000 7f7ef8e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef41630b0 con 0x7f7ef40d2b50 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.836+0000 7f7eeaa00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7ef40b6c70 0x7f7ef415f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47700/0 (socket says 172.21.15.150:47700) 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.836+0000 7f7eeaa00640 1 -- 172.21.15.150:0/777888158 learned_addr learned my addr 172.21.15.150:0/777888158 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.836+0000 7f7ef3400640 1 --2- 172.21.15.150:0/777888158 >> v2:172.21.15.150:3300/0 conn(0x7f7ef415c680 0x7f7ef4162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.837+0000 7f7ef3e00640 1 --2- 172.21.15.150:0/777888158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef40d2b50 0x7f7ef40d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.837+0000 7f7ef2a00640 1 -- 172.21.15.150:0/777888158 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4083641101 0 0) 0x7f7ef40b70d0 con 0x7f7ef40b6c70 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.837+0000 7f7ef2a00640 1 -- 172.21.15.150:0/777888158 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ed8003520 con 0x7f7ef40b6c70 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.837+0000 7f7ef3400640 1 -- 172.21.15.150:0/777888158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef40d2b50 msgr2=0x7f7ef40d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.837+0000 7f7ef3400640 1 --2- 172.21.15.150:0/777888158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef40d2b50 0x7f7ef40d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.837+0000 7f7ef3400640 1 -- 172.21.15.150:0/777888158 >> v1:172.21.15.150:6789/0 conn(0x7f7ef40b6c70 legacy=0x7f7ef415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.837+0000 7f7ef3400640 1 -- 172.21.15.150:0/777888158 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef4163390 con 0x7f7ef415c680 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.838+0000 7f7ef3400640 1 --2- 172.21.15.150:0/777888158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef415c680 0x7f7ef4162730 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40097a0 tx=0x7f7ee402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dabe8f02bfeffb server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.838+0000 7f7ef2a00640 1 -- 172.21.15.150:0/777888158 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4035020 con 0x7f7ef415c680 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.838+0000 7f7ef2a00640 1 -- 172.21.15.150:0/777888158 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee4031810 con 0x7f7ef415c680 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.838+0000 7f7ef2a00640 1 -- 172.21.15.150:0/777888158 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4031be0 con 0x7f7ef415c680 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.839+0000 7f7ef8e00640 1 -- 172.21.15.150:0/777888158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef415c680 msgr2=0x7f7ef4162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.839+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/777888158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef415c680 0x7f7ef4162730 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40097a0 tx=0x7f7ee402b8b0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.839+0000 7f7ef8e00640 1 -- 172.21.15.150:0/777888158 shutdown_connections 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.839+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/777888158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef40d2b50 0x7f7ef40d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.839+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/777888158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef415c680 0x7f7ef4162730 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.839+0000 7f7ef8e00640 1 -- 172.21.15.150:0/777888158 >> 172.21.15.150:0/777888158 conn(0x7f7ef4073480 msgr2=0x7f7ef4073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:41.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.839+0000 7f7ef8e00640 1 -- 172.21.15.150:0/777888158 shutdown_connections 2024-04-15T15:25:41.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.839+0000 7f7ef8e00640 1 -- 172.21.15.150:0/777888158 wait complete. 2024-04-15T15:25:41.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.841+0000 7f7ef8e00640 1 Processor -- start 2024-04-15T15:25:41.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.841+0000 7f7ef8e00640 1 -- start start 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.842+0000 7f7ef8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef40b6c70 0x7f7ef4149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.842+0000 7f7ef8e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ef40d2b50 0x7f7ef40c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.842+0000 7f7ef8e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef415c680 0x7f7ef414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.842+0000 7f7ef8e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef415ff60 con 0x7f7ef40b6c70 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.842+0000 7f7ef8e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef415ca60 con 0x7f7ef40d2b50 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.842+0000 7f7ef8e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef40d1770 con 0x7f7ef415c680 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef40b6c70 0x7f7ef4149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef40b6c70 0x7f7ef4149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36164/0 (socket says 172.21.15.150:36164) 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 -- 172.21.15.150:0/1602195185 learned_addr learned my addr 172.21.15.150:0/1602195185 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:41.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3e00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef415c680 0x7f7ef414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7eeaa00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ef40d2b50 0x7f7ef40c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 -- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef415c680 msgr2=0x7f7ef414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef415c680 0x7f7ef414a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 -- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ef40d2b50 msgr2=0x7f7ef40c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:41.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ef40d2b50 0x7f7ef40c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:41.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 -- 172.21.15.150:0/1602195185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef40c8490 con 0x7f7ef40b6c70 2024-04-15T15:25:41.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.843+0000 7f7ef3400640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef40b6c70 0x7f7ef4149b60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4034040 tx=0x7f7ee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:41.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.844+0000 7f7eebe00640 1 -- 172.21.15.150:0/1602195185 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4040070 con 0x7f7ef40b6c70 2024-04-15T15:25:41.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.844+0000 7f7eebe00640 1 -- 172.21.15.150:0/1602195185 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee4039cf0 con 0x7f7ef40b6c70 2024-04-15T15:25:41.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.844+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef40c8770 con 0x7f7ef40b6c70 2024-04-15T15:25:41.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.844+0000 7f7eebe00640 1 -- 172.21.15.150:0/1602195185 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4035040 con 0x7f7ef40b6c70 2024-04-15T15:25:41.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.844+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef414a0a0 con 0x7f7ef40b6c70 2024-04-15T15:25:41.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.845+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef40755c0 con 0x7f7ef40b6c70 2024-04-15T15:25:41.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.846+0000 7f7eebe00640 1 -- 172.21.15.150:0/1602195185 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7ee4038020 con 0x7f7ef40b6c70 2024-04-15T15:25:41.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.847+0000 7f7eebe00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ed006c980 0x7f7ed006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:41.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.847+0000 7f7eebe00640 1 -- 172.21.15.150:0/1602195185 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f7ee40ad8f0 con 0x7f7ef40b6c70 2024-04-15T15:25:41.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.848+0000 7f7eeaa00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ed006c980 0x7f7ed006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:41.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.849+0000 7f7eeaa00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ed006c980 0x7f7ed006ee40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40ba180 tx=0x7f7edc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:41.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:41.852+0000 7f7eebe00640 1 -- 172.21.15.150:0/1602195185 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ee407bbd0 con 0x7f7ef40b6c70 2024-04-15T15:25:42.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.050+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ef40d3dd0 con 0x7f7ef40b6c70 2024-04-15T15:25:42.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.051+0000 7f7eebe00640 1 -- 172.21.15.150:0/1602195185 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f7ee40abe20 con 0x7f7ef40b6c70 2024-04-15T15:25:42.053 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:25:42.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.053+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ed006c980 msgr2=0x7f7ed006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:42.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.053+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ed006c980 0x7f7ed006ee40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40ba180 tx=0x7f7edc009040 comp rx=0 tx=0).stop 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.053+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef40b6c70 msgr2=0x7f7ef4149b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.053+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef40b6c70 0x7f7ef4149b60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4034040 tx=0x7f7ee40046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.054+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 shutdown_connections 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.054+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f7ed006c980 0x7f7ed006ee40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.054+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ef415c680 0x7f7ef414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.054+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ef40d2b50 0x7f7ef40c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.054+0000 7f7ef8e00640 1 --2- 172.21.15.150:0/1602195185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ef40b6c70 0x7f7ef4149b60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.054+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 >> 172.21.15.150:0/1602195185 conn(0x7f7ef4073480 msgr2=0x7f7ef4160f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.054+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 shutdown_connections 2024-04-15T15:25:42.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.054+0000 7f7ef8e00640 1 -- 172.21.15.150:0/1602195185 wait complete. 2024-04-15T15:25:42.074 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:25:42.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.284+0000 7f6ca5a00640 1 Processor -- start 2024-04-15T15:25:42.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.284+0000 7f6ca5a00640 1 -- start start 2024-04-15T15:25:42.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6ca5a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ca00b43b0 0x7f6ca00aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:42.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6ca5a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca015f6b0 con 0x7f6ca0148110 2024-04-15T15:25:42.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6ca5a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca015ff20 con 0x7f6ca00af1f0 2024-04-15T15:25:42.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6ca5a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca015f960 con 0x7f6ca00b43b0 2024-04-15T15:25:42.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6c97e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ca00b43b0 0x7f6ca00aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:42.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6c97e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ca00b43b0 0x7f6ca00aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36176/0 (socket says 172.21.15.150:36176) 2024-04-15T15:25:42.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6c97e00640 1 -- 172.21.15.150:0/3579874167 learned_addr learned my addr 172.21.15.150:0/3579874167 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:42.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6c97e00640 1 -- 172.21.15.150:0/3579874167 >> v1:172.21.15.187:6789/0 conn(0x7f6ca00af1f0 legacy=0x7f6ca0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:42.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6c97e00640 1 -- 172.21.15.150:0/3579874167 >> v1:172.21.15.150:6789/0 conn(0x7f6ca0148110 legacy=0x7f6ca015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:42.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.286+0000 7f6c97e00640 1 -- 172.21.15.150:0/3579874167 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca015fc40 con 0x7f6ca00b43b0 2024-04-15T15:25:42.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.288+0000 7f6c97e00640 1 --2- 172.21.15.150:0/3579874167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00b43b0 0x7f6ca00aecb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6c880098a0 tx=0x7f6c8802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2cf9499e93ba94 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:42.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.288+0000 7f6c9e000640 1 -- 172.21.15.150:0/3579874167 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88034070 con 0x7f6ca00b43b0 2024-04-15T15:25:42.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.288+0000 7f6c9e000640 1 -- 172.21.15.150:0/3579874167 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c88030840 con 0x7f6ca00b43b0 2024-04-15T15:25:42.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.288+0000 7f6c9e000640 1 -- 172.21.15.150:0/3579874167 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88030c10 con 0x7f6ca00b43b0 2024-04-15T15:25:42.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.289+0000 7f6ca5a00640 1 -- 172.21.15.150:0/3579874167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00b43b0 msgr2=0x7f6ca00aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:42.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.289+0000 7f6ca5a00640 1 --2- 172.21.15.150:0/3579874167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00b43b0 0x7f6ca00aecb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6c880098a0 tx=0x7f6c8802b980 comp rx=0 tx=0).stop 2024-04-15T15:25:42.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.289+0000 7f6ca5a00640 1 -- 172.21.15.150:0/3579874167 shutdown_connections 2024-04-15T15:25:42.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.289+0000 7f6ca5a00640 1 --2- 172.21.15.150:0/3579874167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00b43b0 0x7f6ca00aecb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.289+0000 7f6ca5a00640 1 -- 172.21.15.150:0/3579874167 >> 172.21.15.150:0/3579874167 conn(0x7f6ca00b2460 msgr2=0x7f6ca00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:42.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.289+0000 7f6ca5a00640 1 -- 172.21.15.150:0/3579874167 shutdown_connections 2024-04-15T15:25:42.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.289+0000 7f6ca5a00640 1 -- 172.21.15.150:0/3579874167 wait complete. 2024-04-15T15:25:42.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.291+0000 7f6ca5a00640 1 Processor -- start 2024-04-15T15:25:42.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.291+0000 7f6ca5a00640 1 -- start start 2024-04-15T15:25:42.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6ca5a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00af1f0 0x7f6ca01e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6ca5a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ca00b43b0 0x7f6ca01e8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6ca5a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ca0148110 0x7f6ca015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6ca5a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01484f0 con 0x7f6ca00af1f0 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6ca5a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca00b4e00 con 0x7f6ca00b43b0 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6ca5a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca0148670 con 0x7f6ca0148110 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00af1f0 0x7f6ca01e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00af1f0 0x7f6ca01e9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36182/0 (socket says 172.21.15.150:36182) 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9ea00640 1 -- 172.21.15.150:0/524570052 learned_addr learned my addr 172.21.15.150:0/524570052 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c97e00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ca00b43b0 0x7f6ca01e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9f400640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ca0148110 0x7f6ca015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9ea00640 1 -- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ca0148110 msgr2=0x7f6ca015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9ea00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ca0148110 0x7f6ca015dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9ea00640 1 -- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ca00b43b0 msgr2=0x7f6ca01e8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9ea00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ca00b43b0 0x7f6ca01e8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c9ea00640 1 -- 172.21.15.150:0/524570052 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca01e89c0 con 0x7f6ca00af1f0 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.292+0000 7f6c97e00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ca00b43b0 0x7f6ca01e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.293+0000 7f6c9ea00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00af1f0 0x7f6ca01e9d00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c3c0 tx=0x7f6c9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.293+0000 7f6c96a00640 1 -- 172.21.15.150:0/524570052 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c900052d0 con 0x7f6ca00af1f0 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.293+0000 7f6c96a00640 1 -- 172.21.15.150:0/524570052 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c90005490 con 0x7f6ca00af1f0 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.293+0000 7f6c96a00640 1 -- 172.21.15.150:0/524570052 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90007260 con 0x7f6ca00af1f0 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.293+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca01e8c60 con 0x7f6ca00af1f0 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.293+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca01e9220 con 0x7f6ca00af1f0 2024-04-15T15:25:42.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.294+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca0068c80 con 0x7f6ca00af1f0 2024-04-15T15:25:42.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.295+0000 7f6c96a00640 1 -- 172.21.15.150:0/524570052 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6c90007400 con 0x7f6ca00af1f0 2024-04-15T15:25:42.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.296+0000 7f6c96a00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6c7c06c980 0x7f6c7c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:42.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.296+0000 7f6c96a00640 1 -- 172.21.15.150:0/524570052 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f6c9008b130 con 0x7f6ca00af1f0 2024-04-15T15:25:42.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.297+0000 7f6c97e00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6c7c06c980 0x7f6c7c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:42.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.298+0000 7f6c97e00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6c7c06c980 0x7f6c7c06ee40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c88036000 tx=0x7f6c8803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:42.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.300+0000 7f6c96a00640 1 -- 172.21.15.150:0/524570052 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c900594a0 con 0x7f6ca00af1f0 2024-04-15T15:25:42.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.514+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6ca01e9920 con 0x7f6ca00af1f0 2024-04-15T15:25:42.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.515+0000 7f6c96a00640 1 -- 172.21.15.150:0/524570052 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+124 (secure 0 0 0) 0x7f6c90058da0 con 0x7f6ca00af1f0 2024-04-15T15:25:42.517 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6835/3327940880 2024-04-16T15:25:33.990360+0000 2024-04-15T15:25:42.517 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6833/3327940880 2024-04-16T15:25:33.990360+0000 2024-04-15T15:25:42.517 INFO:teuthology.orchestra.run.smithi150.stderr:listed 2 entries 2024-04-15T15:25:42.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6c7c06c980 msgr2=0x7f6c7c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:42.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6c7c06c980 0x7f6c7c06ee40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c88036000 tx=0x7f6c8803b040 comp rx=0 tx=0).stop 2024-04-15T15:25:42.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00af1f0 msgr2=0x7f6ca01e9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00af1f0 0x7f6ca01e9d00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c3c0 tx=0x7f6c9000c880 comp rx=0 tx=0).stop 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 shutdown_connections 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6c7c06c980 0x7f6c7c06ee40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ca0148110 0x7f6ca015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ca00b43b0 0x7f6ca01e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 --2- 172.21.15.150:0/524570052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca00af1f0 0x7f6ca01e9d00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.517+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 >> 172.21.15.150:0/524570052 conn(0x7f6ca00b2460 msgr2=0x7f6ca00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.518+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 shutdown_connections 2024-04-15T15:25:42.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:42.518+0000 7f6ca5a00640 1 -- 172.21.15.150:0/524570052 wait complete. 2024-04-15T15:25:42.538 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-15T15:25:42.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:42.538 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-15T15:25:42.639 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:25:42.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:42.639 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-15T15:25:42.686 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:25:42.686 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:25:42.686 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:42.744 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:25:42.744 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:42.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:43.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.017+0000 7f417e600640 1 Processor -- start 2024-04-15T15:25:43.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.017+0000 7f417e600640 1 -- start start 2024-04-15T15:25:43.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.018+0000 7f417e600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4178162020 0x7f4178162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.018+0000 7f417e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.018+0000 7f417e600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41780d0090 con 0x7f41780d1ac0 2024-04-15T15:25:43.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417e600640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f41780d1f50 con 0x7f4178162020 2024-04-15T15:25:43.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4178163440 con 0x7f41780cf110 2024-04-15T15:25:43.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417d200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4178162020 0x7f4178162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417d200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4178162020 0x7f4178162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36198/0 (socket says 172.21.15.150:36198) 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417d200640 1 -- 172.21.15.150:0/2244843716 learned_addr learned my addr 172.21.15.150:0/2244843716 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417dc00640 1 --1- 172.21.15.150:0/2244843716 >> v1:172.21.15.150:6789/0 conn(0x7f41780d1ac0 0x7f41780b62a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47718/0 (socket says 172.21.15.150:47718) 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417d200640 1 -- 172.21.15.150:0/2244843716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 msgr2=0x7f41780d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f4177e00640 1 --2- 172.21.15.150:0/2244843716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417d200640 1 --2- 172.21.15.150:0/2244843716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417d200640 1 -- 172.21.15.150:0/2244843716 >> v1:172.21.15.150:6789/0 conn(0x7f41780d1ac0 legacy=0x7f41780b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f417d200640 1 -- 172.21.15.150:0/2244843716 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41780cfad0 con 0x7f4178162020 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f4177400640 1 -- 172.21.15.150:0/2244843716 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819893920 0 0) 0x7f4170001490 con 0x7f41780d1ac0 2024-04-15T15:25:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.019+0000 7f4177e00640 1 --2- 172.21.15.150:0/2244843716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:43.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.021+0000 7f417d200640 1 --2- 172.21.15.150:0/2244843716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178162f00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f416c009870 tx=0x7f416c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6f59de7d2f42d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.021+0000 7f4177400640 1 -- 172.21.15.150:0/2244843716 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c034070 con 0x7f4178162020 2024-04-15T15:25:43.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.021+0000 7f4177400640 1 -- 172.21.15.150:0/2244843716 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c030840 con 0x7f4178162020 2024-04-15T15:25:43.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.021+0000 7f4177400640 1 -- 172.21.15.150:0/2244843716 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c030c10 con 0x7f4178162020 2024-04-15T15:25:43.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.021+0000 7f417e600640 1 -- 172.21.15.150:0/2244843716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 msgr2=0x7f4178162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.021+0000 7f417e600640 1 --2- 172.21.15.150:0/2244843716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178162f00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f416c009870 tx=0x7f416c02b980 comp rx=0 tx=0).stop 2024-04-15T15:25:43.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.021+0000 7f417e600640 1 -- 172.21.15.150:0/2244843716 shutdown_connections 2024-04-15T15:25:43.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.022+0000 7f417e600640 1 --2- 172.21.15.150:0/2244843716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.022+0000 7f417e600640 1 --2- 172.21.15.150:0/2244843716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178162f00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.022+0000 7f417e600640 1 -- 172.21.15.150:0/2244843716 >> 172.21.15.150:0/2244843716 conn(0x7f417806cf00 msgr2=0x7f417806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:43.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.022+0000 7f417e600640 1 -- 172.21.15.150:0/2244843716 shutdown_connections 2024-04-15T15:25:43.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.022+0000 7f417e600640 1 -- 172.21.15.150:0/2244843716 wait complete. 2024-04-15T15:25:43.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.023+0000 7f417e600640 1 Processor -- start 2024-04-15T15:25:43.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.023+0000 7f417e600640 1 -- start start 2024-04-15T15:25:43.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.024+0000 7f417e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.024+0000 7f417e600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41780d1ac0 0x7f41780ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.024+0000 7f417e600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.024+0000 7f417e600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41780cf880 con 0x7f4178162020 2024-04-15T15:25:43.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.024+0000 7f417e600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41780b7b30 con 0x7f41780d1ac0 2024-04-15T15:25:43.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.024+0000 7f417e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41780d2b90 con 0x7f41780cf110 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178140a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36210/0 (socket says 172.21.15.150:36210) 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 -- 172.21.15.150:0/3761121288 learned_addr learned my addr 172.21.15.150:0/3761121288 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f4177e00640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41780d1ac0 0x7f41780ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417d200640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 -- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 msgr2=0x7f41780cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 -- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41780d1ac0 msgr2=0x7f41780ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41780d1ac0 0x7f41780ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 -- 172.21.15.150:0/3761121288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4178140f50 con 0x7f4178162020 2024-04-15T15:25:43.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.025+0000 7f417dc00640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178140a10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f417000c820 tx=0x7f417000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.026+0000 7f4174c00640 1 -- 172.21.15.150:0/3761121288 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170018070 con 0x7f4178162020 2024-04-15T15:25:43.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.026+0000 7f4174c00640 1 -- 172.21.15.150:0/3761121288 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417000a5e0 con 0x7f4178162020 2024-04-15T15:25:43.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.026+0000 7f4174c00640 1 -- 172.21.15.150:0/3761121288 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000a9d0 con 0x7f4178162020 2024-04-15T15:25:43.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.026+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4178143280 con 0x7f4178162020 2024-04-15T15:25:43.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.026+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4178143840 con 0x7f4178162020 2024-04-15T15:25:43.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.027+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41780755c0 con 0x7f4178162020 2024-04-15T15:25:43.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.028+0000 7f4174c00640 1 -- 172.21.15.150:0/3761121288 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f417001b070 con 0x7f4178162020 2024-04-15T15:25:43.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.029+0000 7f4174c00640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f415806c980 0x7f415806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.029+0000 7f4174c00640 1 -- 172.21.15.150:0/3761121288 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f417008b120 con 0x7f4178162020 2024-04-15T15:25:43.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.029+0000 7f417d200640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f415806c980 0x7f415806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.031+0000 7f417d200640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f415806c980 0x7f415806ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f416c036000 tx=0x7f416c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.033+0000 7f4174c00640 1 -- 172.21.15.150:0/3761121288 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41780d0090 con 0x7f4178162020 2024-04-15T15:25:43.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.237+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41780b7650 con 0x7f4178162020 2024-04-15T15:25:43.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.240+0000 7f4174c00640 1 -- 172.21.15.150:0/3761121288 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16848 (secure 0 0 0) 0x7f4170059590 con 0x7f4178162020 2024-04-15T15:25:43.242 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":36,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:33.990388+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/3327940880":"2024-04-16T15:25:33.990360+0000","172.21.15.150:6833/3327940880":"2024-04-16T15:25:33.990360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f415806c980 msgr2=0x7f415806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f415806c980 0x7f415806ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f416c036000 tx=0x7f416c008030 comp rx=0 tx=0).stop 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 msgr2=0x7f4178140a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178140a10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f417000c820 tx=0x7f417000cce0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 shutdown_connections 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4178162020 0x7f4178140a10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41780d1ac0 0x7f41780ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f415806c980 0x7f415806ee40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 --2- 172.21.15.150:0/3761121288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41780cf110 0x7f41780cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 >> 172.21.15.150:0/3761121288 conn(0x7f417806cf00 msgr2=0x7f41780b94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 shutdown_connections 2024-04-15T15:25:43.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.243+0000 7f417e600640 1 -- 172.21.15.150:0/3761121288 wait complete. 2024-04-15T15:25:43.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:25:04.190900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:25:05.324314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:25:43.263 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:25:43.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.471+0000 7f9b81800640 1 Processor -- start 2024-04-15T15:25:43.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.471+0000 7f9b81800640 1 -- start start 2024-04-15T15:25:43.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.472+0000 7f9b81800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9b7c15a440 0x7f9b7c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.472+0000 7f9b81800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c072150 0x7f9b7c0725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.472+0000 7f9b81800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b7c15d520 con 0x7f9b7c06fc80 2024-04-15T15:25:43.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.472+0000 7f9b81800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7c06e280 con 0x7f9b7c15a440 2024-04-15T15:25:43.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.472+0000 7f9b81800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c15d800 con 0x7f9b7c072150 2024-04-15T15:25:43.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.473+0000 7f9b7aa00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9b7c15a440 0x7f9b7c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.473+0000 7f9b7aa00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9b7c15a440 0x7f9b7c15cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36220/0 (socket says 172.21.15.150:36220) 2024-04-15T15:25:43.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.473+0000 7f9b7aa00640 1 -- 172.21.15.150:0/211732469 learned_addr learned my addr 172.21.15.150:0/211732469 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:43.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.473+0000 7f9b7aa00640 1 -- 172.21.15.150:0/211732469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c072150 msgr2=0x7f9b7c0725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:43.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.473+0000 7f9b7b400640 1 --2- 172.21.15.150:0/211732469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c072150 0x7f9b7c0725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.473+0000 7f9b7aa00640 1 --2- 172.21.15.150:0/211732469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c072150 0x7f9b7c0725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.473+0000 7f9b7aa00640 1 -- 172.21.15.150:0/211732469 >> v1:172.21.15.187:6789/0 conn(0x7f9b7c06fc80 legacy=0x7f9b7c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:43.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.473+0000 7f9b7aa00640 1 -- 172.21.15.150:0/211732469 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c15dae0 con 0x7f9b7c15a440 2024-04-15T15:25:43.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.474+0000 7f9b7aa00640 1 --2- 172.21.15.150:0/211732469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c15a440 0x7f9b7c15cef0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c06c9a0 tx=0x7f9b6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffedf23ebdcb159 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b79600640 1 -- 172.21.15.150:0/211732469 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c035020 con 0x7f9b7c15a440 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b79600640 1 -- 172.21.15.150:0/211732469 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c031640 con 0x7f9b7c15a440 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b79600640 1 -- 172.21.15.150:0/211732469 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c02d040 con 0x7f9b7c15a440 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b81800640 1 -- 172.21.15.150:0/211732469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c15a440 msgr2=0x7f9b7c15cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b81800640 1 --2- 172.21.15.150:0/211732469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c15a440 0x7f9b7c15cef0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c06c9a0 tx=0x7f9b6c02b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b81800640 1 -- 172.21.15.150:0/211732469 shutdown_connections 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b81800640 1 --2- 172.21.15.150:0/211732469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c072150 0x7f9b7c0725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b81800640 1 --2- 172.21.15.150:0/211732469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c15a440 0x7f9b7c15cef0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b81800640 1 -- 172.21.15.150:0/211732469 >> 172.21.15.150:0/211732469 conn(0x7f9b7c0b2460 msgr2=0x7f9b7c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:43.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b81800640 1 -- 172.21.15.150:0/211732469 shutdown_connections 2024-04-15T15:25:43.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.475+0000 7f9b81800640 1 -- 172.21.15.150:0/211732469 wait complete. 2024-04-15T15:25:43.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.477+0000 7f9b81800640 1 Processor -- start 2024-04-15T15:25:43.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.477+0000 7f9b81800640 1 -- start start 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.478+0000 7f9b81800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c06fc80 0x7f9b7c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.478+0000 7f9b81800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c072150 0x7f9b7c153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.478+0000 7f9b81800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c15a440 0x7f9b7c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.478+0000 7f9b81800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c15ddc0 con 0x7f9b7c072150 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.478+0000 7f9b81800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c15a820 con 0x7f9b7c06fc80 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.478+0000 7f9b81800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c06f510 con 0x7f9b7c15a440 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c072150 0x7f9b7c153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c072150 0x7f9b7c153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36232/0 (socket says 172.21.15.150:36232) 2024-04-15T15:25:43.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 -- 172.21.15.150:0/1363707299 learned_addr learned my addr 172.21.15.150:0/1363707299 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:43.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7aa00640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c06fc80 0x7f9b7c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7b400640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c15a440 0x7f9b7c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 -- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c15a440 msgr2=0x7f9b7c0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c15a440 0x7f9b7c0b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 -- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c06fc80 msgr2=0x7f9b7c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c06fc80 0x7f9b7c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 -- 172.21.15.150:0/1363707299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c13ae80 con 0x7f9b7c072150 2024-04-15T15:25:43.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.479+0000 7f9b7a000640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c072150 0x7f9b7c153260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800c430 tx=0x7f9b6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.480+0000 7f9b72a00640 1 -- 172.21.15.150:0/1363707299 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68011790 con 0x7f9b7c072150 2024-04-15T15:25:43.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.480+0000 7f9b72a00640 1 -- 172.21.15.150:0/1363707299 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b68017070 con 0x7f9b7c072150 2024-04-15T15:25:43.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.480+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c13f510 con 0x7f9b7c072150 2024-04-15T15:25:43.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.480+0000 7f9b72a00640 1 -- 172.21.15.150:0/1363707299 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68011b70 con 0x7f9b7c072150 2024-04-15T15:25:43.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.480+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c13b0f0 con 0x7f9b7c072150 2024-04-15T15:25:43.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.481+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c068c60 con 0x7f9b7c072150 2024-04-15T15:25:43.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.482+0000 7f9b72a00640 1 -- 172.21.15.150:0/1363707299 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9b68011d10 con 0x7f9b7c072150 2024-04-15T15:25:43.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.483+0000 7f9b72a00640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9b5c06c980 0x7f9b5c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.483+0000 7f9b72a00640 1 -- 172.21.15.150:0/1363707299 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f9b6808bbb0 con 0x7f9b7c072150 2024-04-15T15:25:43.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.483+0000 7f9b7aa00640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9b5c06c980 0x7f9b5c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.485+0000 7f9b7aa00640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9b5c06c980 0x7f9b5c06ee40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c036000 tx=0x7f9b6c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.487+0000 7f9b72a00640 1 -- 172.21.15.150:0/1363707299 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b68059e90 con 0x7f9b7c072150 2024-04-15T15:25:43.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.685+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b7c15ec40 con 0x7f9b7c072150 2024-04-15T15:25:43.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.686+0000 7f9b72a00640 1 -- 172.21.15.150:0/1363707299 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f9b68059790 con 0x7f9b7c072150 2024-04-15T15:25:43.688 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9b5c06c980 msgr2=0x7f9b5c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9b5c06c980 0x7f9b5c06ee40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c036000 tx=0x7f9b6c03c040 comp rx=0 tx=0).stop 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c072150 msgr2=0x7f9b7c153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c072150 0x7f9b7c153260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800c430 tx=0x7f9b6800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 shutdown_connections 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f9b5c06c980 0x7f9b5c06ee40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c15a440 0x7f9b7c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b7c072150 0x7f9b7c153260 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.689+0000 7f9b81800640 1 --2- 172.21.15.150:0/1363707299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c06fc80 0x7f9b7c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.690+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 >> 172.21.15.150:0/1363707299 conn(0x7f9b7c0b2460 msgr2=0x7f9b7c140290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:43.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.690+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 shutdown_connections 2024-04-15T15:25:43.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.690+0000 7f9b81800640 1 -- 172.21.15.150:0/1363707299 wait complete. 2024-04-15T15:25:43.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:25:43.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.924+0000 7f82a0c00640 1 Processor -- start 2024-04-15T15:25:43.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.925+0000 7f82a0c00640 1 -- start start 2024-04-15T15:25:43.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.926+0000 7f82a0c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f829c145070 0x7f829c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.926+0000 7f82a0c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c072690 0x7f829c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.926+0000 7f82a0c00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f829c15ffe0 con 0x7f829c163520 2024-04-15T15:25:43.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.926+0000 7f82a0c00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f829c0afde0 con 0x7f829c145070 2024-04-15T15:25:43.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.926+0000 7f82a0c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f829c1627f0 con 0x7f829c072690 2024-04-15T15:25:43.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f8299e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f829c145070 0x7f829c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f8299400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f829c163520 0x7f829c15f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47722/0 (socket says 172.21.15.150:47722) 2024-04-15T15:25:43.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f8299400640 1 -- 172.21.15.150:0/1016110469 learned_addr learned my addr 172.21.15.150:0/1016110469 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:43.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f829a800640 1 --2- 172.21.15.150:0/1016110469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c072690 0x7f829c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f8298a00640 1 -- 172.21.15.150:0/1016110469 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 282654467 0 0) 0x7f829c15ffe0 con 0x7f829c163520 2024-04-15T15:25:43.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f8299e00640 1 -- 172.21.15.150:0/1016110469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c072690 msgr2=0x7f829c15f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:43.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f8299e00640 1 --2- 172.21.15.150:0/1016110469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c072690 0x7f829c15f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f8299e00640 1 -- 172.21.15.150:0/1016110469 >> v1:172.21.15.150:6789/0 conn(0x7f829c163520 legacy=0x7f829c15f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.927+0000 7f8299e00640 1 -- 172.21.15.150:0/1016110469 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c162a40 con 0x7f829c145070 2024-04-15T15:25:43.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.929+0000 7f8299e00640 1 --2- 172.21.15.150:0/1016110469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c072150 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f828c0097d0 tx=0x7f828c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22fb361eaa51b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f8298a00640 1 -- 172.21.15.150:0/1016110469 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c031770 con 0x7f829c145070 2024-04-15T15:25:43.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f8298a00640 1 -- 172.21.15.150:0/1016110469 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c037070 con 0x7f829c145070 2024-04-15T15:25:43.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f8298a00640 1 -- 172.21.15.150:0/1016110469 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c031b30 con 0x7f829c145070 2024-04-15T15:25:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f82a0c00640 1 -- 172.21.15.150:0/1016110469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 msgr2=0x7f829c072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f82a0c00640 1 --2- 172.21.15.150:0/1016110469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c072150 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f828c0097d0 tx=0x7f828c02b810 comp rx=0 tx=0).stop 2024-04-15T15:25:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f82a0c00640 1 -- 172.21.15.150:0/1016110469 shutdown_connections 2024-04-15T15:25:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f82a0c00640 1 --2- 172.21.15.150:0/1016110469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c072690 0x7f829c15f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f82a0c00640 1 --2- 172.21.15.150:0/1016110469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c072150 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f82a0c00640 1 -- 172.21.15.150:0/1016110469 >> 172.21.15.150:0/1016110469 conn(0x7f829c0b2460 msgr2=0x7f829c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:43.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.930+0000 7f82a0c00640 1 -- 172.21.15.150:0/1016110469 shutdown_connections 2024-04-15T15:25:43.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.931+0000 7f82a0c00640 1 -- 172.21.15.150:0/1016110469 wait complete. 2024-04-15T15:25:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.932+0000 7f82a0c00640 1 Processor -- start 2024-04-15T15:25:43.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.932+0000 7f82a0c00640 1 -- start start 2024-04-15T15:25:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.933+0000 7f82a0c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c072690 0x7f829c13c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.933+0000 7f82a0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c13f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.933+0000 7f82a0c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c163520 0x7f829c1405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.933+0000 7f82a0c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f829c145450 con 0x7f829c145070 2024-04-15T15:25:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.933+0000 7f82a0c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f829c163000 con 0x7f829c072690 2024-04-15T15:25:43.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.933+0000 7f82a0c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f829c0aedd0 con 0x7f829c163520 2024-04-15T15:25:43.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c13f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c13f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36254/0 (socket says 172.21.15.150:36254) 2024-04-15T15:25:43.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 -- 172.21.15.150:0/3887811749 learned_addr learned my addr 172.21.15.150:0/3887811749 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299e00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c072690 0x7f829c13c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 -- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c163520 msgr2=0x7f829c1405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c163520 0x7f829c1405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 -- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c072690 msgr2=0x7f829c13c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c072690 0x7f829c13c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 -- 172.21.15.150:0/3887811749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c13e490 con 0x7f829c145070 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.934+0000 7f8299400640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c13f530 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f828400c430 tx=0x7f828400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.935+0000 7f8292000640 1 -- 172.21.15.150:0/3887811749 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840074e0 con 0x7f829c145070 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.935+0000 7f8292000640 1 -- 172.21.15.150:0/3887811749 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82840047e0 con 0x7f829c145070 2024-04-15T15:25:43.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.935+0000 7f8292000640 1 -- 172.21.15.150:0/3887811749 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284004bd0 con 0x7f829c145070 2024-04-15T15:25:43.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.935+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c13e770 con 0x7f829c145070 2024-04-15T15:25:43.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.935+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c13ed30 con 0x7f829c145070 2024-04-15T15:25:43.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.936+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829c068c80 con 0x7f829c145070 2024-04-15T15:25:43.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.937+0000 7f8292000640 1 -- 172.21.15.150:0/3887811749 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8284011070 con 0x7f829c145070 2024-04-15T15:25:43.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.938+0000 7f8292000640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f827806c9b0 0x7f827806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:43.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.938+0000 7f8292000640 1 -- 172.21.15.150:0/3887811749 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f828408b8d0 con 0x7f829c145070 2024-04-15T15:25:43.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.938+0000 7f8299e00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f827806c9b0 0x7f827806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:43.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.939+0000 7f8299e00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f827806c9b0 0x7f827806ee70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f828c036000 tx=0x7f828c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:43.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:43.942+0000 7f8292000640 1 -- 172.21.15.150:0/3887811749 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8284059c40 con 0x7f829c145070 2024-04-15T15:25:44.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.149+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f829c0b2080 con 0x7f829c145070 2024-04-15T15:25:44.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.149+0000 7f8292000640 1 -- 172.21.15.150:0/3887811749 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f8284059540 con 0x7f829c145070 2024-04-15T15:25:44.151 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f827806c9b0 msgr2=0x7f827806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f827806c9b0 0x7f827806ee70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f828c036000 tx=0x7f828c008030 comp rx=0 tx=0).stop 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 msgr2=0x7f829c13f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c13f530 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f828400c430 tx=0x7f828400c8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 shutdown_connections 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f827806c9b0 0x7f827806ee70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f829c163520 0x7f829c1405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f829c145070 0x7f829c13f530 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 --2- 172.21.15.150:0/3887811749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c072690 0x7f829c13c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 >> 172.21.15.150:0/3887811749 conn(0x7f829c0b2460 msgr2=0x7f829c0cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:44.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 shutdown_connections 2024-04-15T15:25:44.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.152+0000 7f82a0c00640 1 -- 172.21.15.150:0/3887811749 wait complete. 2024-04-15T15:25:44.173 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:25:44.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.387+0000 7ff7f9200640 1 Processor -- start 2024-04-15T15:25:44.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.387+0000 7ff7f9200640 1 -- start start 2024-04-15T15:25:44.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.389+0000 7ff7f9200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 0x7ff7f40b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.389+0000 7ff7f9200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7f406f260 0x7ff7f406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.389+0000 7ff7f9200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f40b4410 con 0x7ff7f4145170 2024-04-15T15:25:44.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.389+0000 7ff7f9200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f4145550 con 0x7ff7f406f260 2024-04-15T15:25:44.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.389+0000 7ff7f9200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f40b46f0 con 0x7ff7f40b54b0 2024-04-15T15:25:44.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.389+0000 7ff7f2800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7f406f260 0x7ff7f406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.390+0000 7ff7f2800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7f406f260 0x7ff7f406f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36268/0 (socket says 172.21.15.150:36268) 2024-04-15T15:25:44.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.390+0000 7ff7f2800640 1 -- 172.21.15.150:0/3867295034 learned_addr learned my addr 172.21.15.150:0/3867295034 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:44.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.390+0000 7ff7f1400640 1 --2- 172.21.15.150:0/3867295034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 0x7ff7f40b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.390+0000 7ff7f2800640 1 -- 172.21.15.150:0/3867295034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 msgr2=0x7ff7f40b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:44.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.390+0000 7ff7f2800640 1 --2- 172.21.15.150:0/3867295034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 0x7ff7f40b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.390+0000 7ff7f2800640 1 -- 172.21.15.150:0/3867295034 >> v1:172.21.15.187:6789/0 conn(0x7ff7f4145170 legacy=0x7ff7f40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:44.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.390+0000 7ff7f2800640 1 -- 172.21.15.150:0/3867295034 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f40b49d0 con 0x7ff7f406f260 2024-04-15T15:25:44.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.392+0000 7ff7f2800640 1 --2- 172.21.15.150:0/3867295034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f406f260 0x7ff7f406f6f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009e50 tx=0x7ff7ec02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee154691b57860c server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:44.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.392+0000 7ff7f0a00640 1 -- 172.21.15.150:0/3867295034 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec02cb00 con 0x7ff7f406f260 2024-04-15T15:25:44.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.392+0000 7ff7f0a00640 1 -- 172.21.15.150:0/3867295034 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec02cca0 con 0x7ff7f406f260 2024-04-15T15:25:44.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.392+0000 7ff7f0a00640 1 -- 172.21.15.150:0/3867295034 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec034610 con 0x7ff7f406f260 2024-04-15T15:25:44.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.393+0000 7ff7f9200640 1 -- 172.21.15.150:0/3867295034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f406f260 msgr2=0x7ff7f406f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.393+0000 7ff7f9200640 1 --2- 172.21.15.150:0/3867295034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f406f260 0x7ff7f406f6f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009e50 tx=0x7ff7ec02bf60 comp rx=0 tx=0).stop 2024-04-15T15:25:44.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.393+0000 7ff7f9200640 1 -- 172.21.15.150:0/3867295034 shutdown_connections 2024-04-15T15:25:44.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.393+0000 7ff7f9200640 1 --2- 172.21.15.150:0/3867295034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f406f260 0x7ff7f406f6f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.393+0000 7ff7f9200640 1 --2- 172.21.15.150:0/3867295034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 0x7ff7f40b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.393+0000 7ff7f9200640 1 -- 172.21.15.150:0/3867295034 >> 172.21.15.150:0/3867295034 conn(0x7ff7f40b2460 msgr2=0x7ff7f40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:44.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.393+0000 7ff7f9200640 1 -- 172.21.15.150:0/3867295034 shutdown_connections 2024-04-15T15:25:44.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.393+0000 7ff7f9200640 1 -- 172.21.15.150:0/3867295034 wait complete. 2024-04-15T15:25:44.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.395+0000 7ff7f9200640 1 Processor -- start 2024-04-15T15:25:44.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.395+0000 7ff7f9200640 1 -- start start 2024-04-15T15:25:44.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f9200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f406f260 0x7ff7f4160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f9200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 0x7ff7f41421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f9200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f4145170 0x7ff7f4142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f9200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40ce240 con 0x7ff7f4145170 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f9200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40b4f90 con 0x7ff7f406f260 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f9200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f40b4030 con 0x7ff7f40b54b0 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f2800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f4145170 0x7ff7f4142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f2800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f4145170 0x7ff7f4142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36272/0 (socket says 172.21.15.150:36272) 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f2800640 1 -- 172.21.15.150:0/1979992837 learned_addr learned my addr 172.21.15.150:0/1979992837 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f1e00640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f406f260 0x7ff7f4160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.396+0000 7ff7f1400640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 0x7ff7f41421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7f2800640 1 -- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 msgr2=0x7ff7f41421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7f2800640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 0x7ff7f41421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7f2800640 1 -- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f406f260 msgr2=0x7ff7f4160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7f2800640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f406f260 0x7ff7f4160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7f2800640 1 -- 172.21.15.150:0/1979992837 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f413cda0 con 0x7ff7f4145170 2024-04-15T15:25:44.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7f2800640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f4145170 0x7ff7f4142720 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009f70 tx=0x7ff7ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:44.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7ea000640 1 -- 172.21.15.150:0/1979992837 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec043020 con 0x7ff7f4145170 2024-04-15T15:25:44.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7ea000640 1 -- 172.21.15.150:0/1979992837 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec02ce50 con 0x7ff7f4145170 2024-04-15T15:25:44.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7ea000640 1 -- 172.21.15.150:0/1979992837 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec009510 con 0x7ff7f4145170 2024-04-15T15:25:44.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.397+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f413bd20 con 0x7ff7f4145170 2024-04-15T15:25:44.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.398+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f413c2c0 con 0x7ff7f4145170 2024-04-15T15:25:44.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.399+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f4068c80 con 0x7ff7f4145170 2024-04-15T15:25:44.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.400+0000 7ff7ea000640 1 -- 172.21.15.150:0/1979992837 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff7ec046020 con 0x7ff7f4145170 2024-04-15T15:25:44.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.400+0000 7ff7ea000640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d006c980 0x7ff7d006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.401+0000 7ff7ea000640 1 -- 172.21.15.150:0/1979992837 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7ff7ec0ae290 con 0x7ff7f4145170 2024-04-15T15:25:44.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.401+0000 7ff7f1e00640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d006c980 0x7ff7d006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.402+0000 7ff7f1e00640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d006c980 0x7ff7d006ee40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4007fd0 tx=0x7ff7e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:44.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.405+0000 7ff7ea000640 1 -- 172.21.15.150:0/1979992837 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7ec04d030 con 0x7ff7f4145170 2024-04-15T15:25:44.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.603+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7f413d990 con 0x7ff7f4145170 2024-04-15T15:25:44.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.604+0000 7ff7ea000640 1 -- 172.21.15.150:0/1979992837 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7ff7ec07c640 con 0x7ff7f4145170 2024-04-15T15:25:44.605 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:25:44.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d006c980 msgr2=0x7ff7d006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d006c980 0x7ff7d006ee40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4007fd0 tx=0x7ff7e4009040 comp rx=0 tx=0).stop 2024-04-15T15:25:44.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f4145170 msgr2=0x7ff7f4142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f4145170 0x7ff7f4142720 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009f70 tx=0x7ff7ec004a40 comp rx=0 tx=0).stop 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 shutdown_connections 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f4145170 0x7ff7f4142720 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d006c980 0x7ff7d006ee40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7f40b54b0 0x7ff7f41421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 --2- 172.21.15.150:0/1979992837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7f406f260 0x7ff7f4160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.606+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 >> 172.21.15.150:0/1979992837 conn(0x7ff7f40b2460 msgr2=0x7ff7f40ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.607+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 shutdown_connections 2024-04-15T15:25:44.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.607+0000 7ff7f9200640 1 -- 172.21.15.150:0/1979992837 wait complete. 2024-04-15T15:25:44.627 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:25:44.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.823+0000 7f4f5e400640 1 Processor -- start 2024-04-15T15:25:44.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.823+0000 7f4f5e400640 1 -- start start 2024-04-15T15:25:44.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.825+0000 7f4f5e400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4f580b43b0 0x7f4f580aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.825+0000 7f4f5e400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4f580af1f0 0x7f4f58072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.825+0000 7f4f5e400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f5815f6b0 con 0x7f4f58148110 2024-04-15T15:25:44.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.825+0000 7f4f5e400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f4f5815ff20 con 0x7f4f580af1f0 2024-04-15T15:25:44.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.825+0000 7f4f5e400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4f5815f960 con 0x7f4f580b43b0 2024-04-15T15:25:44.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.825+0000 7f4f57e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4f580af1f0 0x7f4f58072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.826+0000 7f4f57e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4f580af1f0 0x7f4f58072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36284/0 (socket says 172.21.15.150:36284) 2024-04-15T15:25:44.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.826+0000 7f4f57e00640 1 -- 172.21.15.150:0/1380767406 learned_addr learned my addr 172.21.15.150:0/1380767406 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:44.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.826+0000 7f4f4ea00640 1 --2- 172.21.15.150:0/1380767406 >> v2:172.21.15.187:3300/0 conn(0x7f4f580b43b0 0x7f4f580aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.826+0000 7f4f57e00640 1 -- 172.21.15.150:0/1380767406 >> v2:172.21.15.187:3300/0 conn(0x7f4f580b43b0 msgr2=0x7f4f580aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:44.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.826+0000 7f4f57e00640 1 --2- 172.21.15.150:0/1380767406 >> v2:172.21.15.187:3300/0 conn(0x7f4f580b43b0 0x7f4f580aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.826+0000 7f4f57e00640 1 -- 172.21.15.150:0/1380767406 >> v1:172.21.15.187:6789/0 conn(0x7f4f58148110 legacy=0x7f4f5815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:44.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.826+0000 7f4f57e00640 1 -- 172.21.15.150:0/1380767406 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5815fc40 con 0x7f4f580af1f0 2024-04-15T15:25:44.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.827+0000 7f4f57e00640 1 --2- 172.21.15.150:0/1380767406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580af1f0 0x7f4f58072100 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009e50 tx=0x7f4f5002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ae71c388a444e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:44.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.827+0000 7f4f56a00640 1 -- 172.21.15.150:0/1380767406 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5002ca80 con 0x7f4f580af1f0 2024-04-15T15:25:44.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.827+0000 7f4f56a00640 1 -- 172.21.15.150:0/1380767406 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5002cc20 con 0x7f4f580af1f0 2024-04-15T15:25:44.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.827+0000 7f4f56a00640 1 -- 172.21.15.150:0/1380767406 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f500343f0 con 0x7f4f580af1f0 2024-04-15T15:25:44.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.828+0000 7f4f5e400640 1 -- 172.21.15.150:0/1380767406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580af1f0 msgr2=0x7f4f58072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.828+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1380767406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580af1f0 0x7f4f58072100 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009e50 tx=0x7f4f5002bee0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.828+0000 7f4f5e400640 1 -- 172.21.15.150:0/1380767406 shutdown_connections 2024-04-15T15:25:44.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.828+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1380767406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580af1f0 0x7f4f58072100 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.828+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1380767406 >> v2:172.21.15.187:3300/0 conn(0x7f4f580b43b0 0x7f4f580aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.828+0000 7f4f5e400640 1 -- 172.21.15.150:0/1380767406 >> 172.21.15.150:0/1380767406 conn(0x7f4f580b2460 msgr2=0x7f4f580b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:44.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.828+0000 7f4f5e400640 1 -- 172.21.15.150:0/1380767406 shutdown_connections 2024-04-15T15:25:44.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.828+0000 7f4f5e400640 1 -- 172.21.15.150:0/1380767406 wait complete. 2024-04-15T15:25:44.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.830+0000 7f4f5e400640 1 Processor -- start 2024-04-15T15:25:44.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.830+0000 7f4f5e400640 1 -- start start 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f5e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f580af1f0 0x7f4f5815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f5e400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580b43b0 0x7f4f581e5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f5e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f58148110 0x7f4f58144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f5e400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581484f0 con 0x7f4f580b43b0 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f5e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f580b4e00 con 0x7f4f58148110 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f5e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4f58148670 con 0x7f4f580af1f0 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f4ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580b43b0 0x7f4f581e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f4ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580b43b0 0x7f4f581e5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36290/0 (socket says 172.21.15.150:36290) 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.831+0000 7f4f4ea00640 1 -- 172.21.15.150:0/1728907602 learned_addr learned my addr 172.21.15.150:0/1728907602 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:44.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f57e00640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f58148110 0x7f4f58144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4ea00640 1 -- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f580af1f0 msgr2=0x7f4f5815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4ea00640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f580af1f0 0x7f4f5815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4ea00640 1 -- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f58148110 msgr2=0x7f4f58144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4ea00640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f58148110 0x7f4f58144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4ea00640 1 -- 172.21.15.150:0/1728907602 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5815ebc0 con 0x7f4f580b43b0 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4ea00640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580b43b0 0x7f4f581e5860 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000d800 tx=0x7f4f4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4fe00640 1 -- 172.21.15.150:0/1728907602 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40004030 con 0x7f4f580b43b0 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4fe00640 1 -- 172.21.15.150:0/1728907602 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f400041d0 con 0x7f4f580b43b0 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f4fe00640 1 -- 172.21.15.150:0/1728907602 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000b6c0 con 0x7f4f580b43b0 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f58133570 con 0x7f4f580b43b0 2024-04-15T15:25:44.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.832+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f58133b30 con 0x7f4f580b43b0 2024-04-15T15:25:44.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.833+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f58134240 con 0x7f4f580b43b0 2024-04-15T15:25:44.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.834+0000 7f4f4fe00640 1 -- 172.21.15.150:0/1728907602 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4f4000b860 con 0x7f4f580b43b0 2024-04-15T15:25:44.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.835+0000 7f4f4fe00640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f3406c980 0x7f4f3406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:44.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.835+0000 7f4f4fe00640 1 -- 172.21.15.150:0/1728907602 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f4f4008bfc0 con 0x7f4f580b43b0 2024-04-15T15:25:44.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.836+0000 7f4f57400640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f3406c980 0x7f4f3406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:44.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.837+0000 7f4f57400640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f3406c980 0x7f4f3406ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4f48003cc0 tx=0x7f4f480079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:44.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:44.840+0000 7f4f4fe00640 1 -- 172.21.15.150:0/1728907602 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f5815ebc0 con 0x7f4f580b43b0 2024-04-15T15:25:45.041 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.040+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4f58068c80 con 0x7f4f580b43b0 2024-04-15T15:25:45.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.041+0000 7f4f4fe00640 1 -- 172.21.15.150:0/1728907602 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+124 (secure 0 0 0) 0x7f4f4001c360 con 0x7f4f580b43b0 2024-04-15T15:25:45.042 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6835/3327940880 2024-04-16T15:25:33.990360+0000 2024-04-15T15:25:45.043 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6833/3327940880 2024-04-16T15:25:33.990360+0000 2024-04-15T15:25:45.043 INFO:teuthology.orchestra.run.smithi150.stderr:listed 2 entries 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f3406c980 msgr2=0x7f4f3406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f3406c980 0x7f4f3406ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4f48003cc0 tx=0x7f4f480079b0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580b43b0 msgr2=0x7f4f581e5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580b43b0 0x7f4f581e5860 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000d800 tx=0x7f4f4000dcc0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 shutdown_connections 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f58148110 0x7f4f58144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f580b43b0 0x7f4f581e5860 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f4f3406c980 0x7f4f3406ee40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 --2- 172.21.15.150:0/1728907602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f580af1f0 0x7f4f5815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 >> 172.21.15.150:0/1728907602 conn(0x7f4f580b2460 msgr2=0x7f4f580cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:45.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 shutdown_connections 2024-04-15T15:25:45.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.043+0000 7f4f5e400640 1 -- 172.21.15.150:0/1728907602 wait complete. 2024-04-15T15:25:45.062 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-15T15:25:45.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:45.062 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-15T15:25:45.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:25:45.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:45.178 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-15T15:25:45.225 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:25:45.226 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:25:45.226 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:45.283 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:25:45.283 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:45.338 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:45.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.546+0000 7fe98f200640 1 Processor -- start 2024-04-15T15:25:45.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.546+0000 7fe98f200640 1 -- start start 2024-04-15T15:25:45.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.548+0000 7fe98f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe990148130 0x7fe99015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.548+0000 7fe98f200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe990162380 0x7fe990071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.548+0000 7fe98f200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe990162940 con 0x7fe990072490 2024-04-15T15:25:45.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.548+0000 7fe98f200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe990148510 con 0x7fe990162380 2024-04-15T15:25:45.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.548+0000 7fe98f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe99015fd70 con 0x7fe990148130 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.548+0000 7fe98e800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe990072490 0x7fe99006dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47734/0 (socket says 172.21.15.150:47734) 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.548+0000 7fe98e800640 1 -- 172.21.15.150:0/326646527 learned_addr learned my addr 172.21.15.150:0/326646527 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98d400640 1 --2- 172.21.15.150:0/326646527 >> v2:172.21.15.187:3300/0 conn(0x7fe990162380 0x7fe990071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98de00640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe990148130 0x7fe99015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325633005 0 0) 0x7fe990162940 con 0x7fe990072490 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe974003520 con 0x7fe990072490 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3377780291 0 0) 0x7fe974003520 con 0x7fe990072490 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe990148130 msgr2=0x7fe99015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe990148130 0x7fe99015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 >> v2:172.21.15.187:3300/0 conn(0x7fe990162380 msgr2=0x7fe990071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 --2- 172.21.15.150:0/326646527 >> v2:172.21.15.187:3300/0 conn(0x7fe990162380 0x7fe990071f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990162c20 con 0x7fe990072490 2024-04-15T15:25:45.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fe9880019f0 con 0x7fe990072490 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 >> v1:172.21.15.150:6789/0 conn(0x7fe990072490 legacy=0x7fe99006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9740039b0 0x7fe974003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe974004800 0x7fe974004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe990148130 0x7fe9740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe990148510 con 0x7fe974004800 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe99015fd70 con 0x7fe9740039b0 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.549+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe97400b9c0 con 0x7fe990148130 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.550+0000 7fe98d400640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe974004800 0x7fe974004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.550+0000 7fe98de00640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9740039b0 0x7fe974003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.550+0000 7fe98d400640 1 -- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe990148130 msgr2=0x7fe9740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.550+0000 7fe98d400640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe990148130 0x7fe9740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.550+0000 7fe98d400640 1 -- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9740039b0 msgr2=0x7fe974003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.550+0000 7fe98d400640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9740039b0 0x7fe974003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.550+0000 7fe98d400640 1 -- 172.21.15.150:0/326646527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe974003520 con 0x7fe974004800 2024-04-15T15:25:45.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.550+0000 7fe98de00640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9740039b0 0x7fe974003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.551+0000 7fe98d400640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe974004800 0x7fe974004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009ac0 tx=0x7fe974005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e39043969a712f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.551+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c039070 con 0x7fe974004800 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c02c710 con 0x7fe974004800 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98ca00640 1 -- 172.21.15.150:0/326646527 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c009690 con 0x7fe974004800 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98f200640 1 -- 172.21.15.150:0/326646527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe974004800 msgr2=0x7fe974004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98f200640 1 --2- 172.21.15.150:0/326646527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe974004800 0x7fe974004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009ac0 tx=0x7fe974005830 comp rx=0 tx=0).stop 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98e800640 1 -- 172.21.15.150:0/326646527 reap_dead start 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98f200640 1 -- 172.21.15.150:0/326646527 shutdown_connections 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98f200640 1 -- 172.21.15.150:0/326646527 >> 172.21.15.150:0/326646527 conn(0x7fe99006b930 msgr2=0x7fe99006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98f200640 1 -- 172.21.15.150:0/326646527 shutdown_connections 2024-04-15T15:25:45.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.552+0000 7fe98f200640 1 -- 172.21.15.150:0/326646527 wait complete. 2024-04-15T15:25:45.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.554+0000 7fe98f200640 1 Processor -- start 2024-04-15T15:25:45.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.554+0000 7fe98f200640 1 -- start start 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.555+0000 7fe98f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe974004800 0x7fe99015e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.555+0000 7fe98f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9901459e0 0x7fe990144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.555+0000 7fe98f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9900ce090 0x7fe9900ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.555+0000 7fe98f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900aeb90 con 0x7fe9900ce090 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.555+0000 7fe98f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901631e0 con 0x7fe974004800 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.555+0000 7fe98f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900aed10 con 0x7fe9901459e0 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.555+0000 7fe98e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9900ce090 0x7fe9900ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9900ce090 0x7fe9900ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36316/0 (socket says 172.21.15.150:36316) 2024-04-15T15:25:45.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98e800640 1 -- 172.21.15.150:0/1073091167 learned_addr learned my addr 172.21.15.150:0/1073091167 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98de00640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe974004800 0x7fe99015e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98d400640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9901459e0 0x7fe990144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98e800640 1 -- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9901459e0 msgr2=0x7fe990144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98e800640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9901459e0 0x7fe990144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98e800640 1 -- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe974004800 msgr2=0x7fe99015e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98e800640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe974004800 0x7fe99015e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98e800640 1 -- 172.21.15.150:0/1073091167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990146430 con 0x7fe9900ce090 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98de00640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe974004800 0x7fe99015e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe98e800640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9900ce090 0x7fe9900ce540 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe9900ae620 tx=0x7fe98800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.556+0000 7fe986000640 1 -- 172.21.15.150:0/1073091167 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98800a5b0 con 0x7fe9900ce090 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.557+0000 7fe986000640 1 -- 172.21.15.150:0/1073091167 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe98800a750 con 0x7fe9900ce090 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.557+0000 7fe986000640 1 -- 172.21.15.150:0/1073091167 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98800ab00 con 0x7fe9900ce090 2024-04-15T15:25:45.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.557+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe990146710 con 0x7fe9900ce090 2024-04-15T15:25:45.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.557+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe990146cd0 con 0x7fe9900ce090 2024-04-15T15:25:45.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.558+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9900755c0 con 0x7fe9900ce090 2024-04-15T15:25:45.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.559+0000 7fe986000640 1 -- 172.21.15.150:0/1073091167 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe98800aca0 con 0x7fe9900ce090 2024-04-15T15:25:45.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.559+0000 7fe986000640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe97006c980 0x7fe97006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:45.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.560+0000 7fe986000640 1 -- 172.21.15.150:0/1073091167 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fe98808b4f0 con 0x7fe9900ce090 2024-04-15T15:25:45.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.560+0000 7fe98de00640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe97006c980 0x7fe97006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:45.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.561+0000 7fe98de00640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe97006c980 0x7fe97006ee40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe9900ced00 tx=0x7fe980008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:45.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.564+0000 7fe986000640 1 -- 172.21.15.150:0/1073091167 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe98801a020 con 0x7fe9900ce090 2024-04-15T15:25:45.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.768+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9900adbe0 con 0x7fe9900ce090 2024-04-15T15:25:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.771+0000 7fe986000640 1 -- 172.21.15.150:0/1073091167 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16848 (secure 0 0 0) 0x7fe98800f260 con 0x7fe9900ce090 2024-04-15T15:25:45.773 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:45.773 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":36,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:33.990388+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/3327940880":"2024-04-16T15:25:33.990360+0000","172.21.15.150:6833/3327940880":"2024-04-16T15:25:33.990360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.773+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe97006c980 msgr2=0x7fe97006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.773+0000 7fe98f200640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe97006c980 0x7fe97006ee40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe9900ced00 tx=0x7fe980008040 comp rx=0 tx=0).stop 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.773+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9900ce090 msgr2=0x7fe9900ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.773+0000 7fe98f200640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9900ce090 0x7fe9900ce540 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe9900ae620 tx=0x7fe98800ccd0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.774+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 shutdown_connections 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.774+0000 7fe98f200640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9900ce090 0x7fe9900ce540 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.774+0000 7fe98f200640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe97006c980 0x7fe97006ee40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.774+0000 7fe98f200640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9901459e0 0x7fe990144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.774+0000 7fe98f200640 1 --2- 172.21.15.150:0/1073091167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe974004800 0x7fe99015e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.774+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 >> 172.21.15.150:0/1073091167 conn(0x7fe99006b930 msgr2=0x7fe99006e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.774+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 shutdown_connections 2024-04-15T15:25:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:45.774+0000 7fe98f200640 1 -- 172.21.15.150:0/1073091167 wait complete. 2024-04-15T15:25:45.795 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:25:04.190900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:25:05.324314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:25:45.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:25:46.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.013+0000 7f8727e00640 1 Processor -- start 2024-04-15T15:25:46.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.013+0000 7f8727e00640 1 -- start start 2024-04-15T15:25:46.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8727e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f872815d8f0 0x7f87280ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8727e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87280ce850 0x7f87280b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8727e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87280cecd0 con 0x7f872815d510 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8727e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f87280cefb0 con 0x7f872815d8f0 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8727e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f872806dfa0 con 0x7f87280ce850 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8726000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f872815d8f0 0x7f87280ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8726000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f872815d8f0 0x7f87280ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36328/0 (socket says 172.21.15.150:36328) 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8726000640 1 -- 172.21.15.150:0/3406222915 learned_addr learned my addr 172.21.15.150:0/3406222915 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8726000640 1 -- 172.21.15.150:0/3406222915 >> v2:172.21.15.187:3300/0 conn(0x7f87280ce850 msgr2=0x7f87280b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:46.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8726000640 1 --2- 172.21.15.150:0/3406222915 >> v2:172.21.15.187:3300/0 conn(0x7f87280ce850 0x7f87280b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.015+0000 7f8726000640 1 -- 172.21.15.150:0/3406222915 >> v1:172.21.15.187:6789/0 conn(0x7f872815d510 legacy=0x7f872806ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.016+0000 7f8726000640 1 -- 172.21.15.150:0/3406222915 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872806e280 con 0x7f872815d8f0 2024-04-15T15:25:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.017+0000 7f8726000640 1 --2- 172.21.15.150:0/3406222915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f872815d8f0 0x7f87280ce310 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f87280703b0 tx=0x7f871002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f6be715ac937b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.017+0000 7f8725600640 1 -- 172.21.15.150:0/3406222915 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871002ca80 con 0x7f872815d8f0 2024-04-15T15:25:46.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.017+0000 7f8725600640 1 -- 172.21.15.150:0/3406222915 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871002cc20 con 0x7f872815d8f0 2024-04-15T15:25:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.017+0000 7f8725600640 1 -- 172.21.15.150:0/3406222915 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100343f0 con 0x7f872815d8f0 2024-04-15T15:25:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.018+0000 7f8727e00640 1 -- 172.21.15.150:0/3406222915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f872815d8f0 msgr2=0x7f87280ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.018+0000 7f8727e00640 1 --2- 172.21.15.150:0/3406222915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f872815d8f0 0x7f87280ce310 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f87280703b0 tx=0x7f871002bee0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.018+0000 7f8727e00640 1 -- 172.21.15.150:0/3406222915 shutdown_connections 2024-04-15T15:25:46.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.018+0000 7f8727e00640 1 --2- 172.21.15.150:0/3406222915 >> v2:172.21.15.187:3300/0 conn(0x7f87280ce850 0x7f87280b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.018+0000 7f8727e00640 1 --2- 172.21.15.150:0/3406222915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f872815d8f0 0x7f87280ce310 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.018+0000 7f8727e00640 1 -- 172.21.15.150:0/3406222915 >> 172.21.15.150:0/3406222915 conn(0x7f87280b23a0 msgr2=0x7f87280b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:46.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.018+0000 7f8727e00640 1 -- 172.21.15.150:0/3406222915 shutdown_connections 2024-04-15T15:25:46.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.018+0000 7f8727e00640 1 -- 172.21.15.150:0/3406222915 wait complete. 2024-04-15T15:25:46.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.020+0000 7f8727e00640 1 Processor -- start 2024-04-15T15:25:46.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.020+0000 7f8727e00640 1 -- start start 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8727e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87280ce850 0x7f87280aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8727e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872815d510 0x7f87281e3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8727e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8728146e10 0x7f87280b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8727e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f87280b6070 con 0x7f87280ce850 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8727e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87280b5ef0 con 0x7f8728146e10 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8727e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f87280b61f0 con 0x7f872815d510 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8726a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87280ce850 0x7f87280aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8726a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87280ce850 0x7f87280aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36334/0 (socket says 172.21.15.150:36334) 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8726a00640 1 -- 172.21.15.150:0/4058452718 learned_addr learned my addr 172.21.15.150:0/4058452718 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.021+0000 7f8726000640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872815d510 0x7f87281e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8726a00640 1 -- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872815d510 msgr2=0x7f87281e3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8726a00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872815d510 0x7f87281e3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8727400640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8728146e10 0x7f87280b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8726a00640 1 -- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8728146e10 msgr2=0x7f87280b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8726a00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8728146e10 0x7f87280b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8726a00640 1 -- 172.21.15.150:0/4058452718 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87281e3c50 con 0x7f87280ce850 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8726000640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872815d510 0x7f87281e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8726a00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87280ce850 0x7f87280aee70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f871800c430 tx=0x7f871800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f871ea00640 1 -- 172.21.15.150:0/4058452718 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718011790 con 0x7f87280ce850 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f871ea00640 1 -- 172.21.15.150:0/4058452718 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8718017070 con 0x7f87280ce850 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f871ea00640 1 -- 172.21.15.150:0/4058452718 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718011b70 con 0x7f87280ce850 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87281e3f30 con 0x7f87280ce850 2024-04-15T15:25:46.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.022+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87281e44f0 con 0x7f87280ce850 2024-04-15T15:25:46.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.024+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87280755c0 con 0x7f87280ce850 2024-04-15T15:25:46.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.025+0000 7f871ea00640 1 -- 172.21.15.150:0/4058452718 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8718011d10 con 0x7f87280ce850 2024-04-15T15:25:46.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.025+0000 7f871ea00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f870406c980 0x7f870406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.025+0000 7f871ea00640 1 -- 172.21.15.150:0/4058452718 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f871808bac0 con 0x7f87280ce850 2024-04-15T15:25:46.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.026+0000 7f8726000640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f870406c980 0x7f870406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.027+0000 7f8726000640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f870406c980 0x7f870406ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8710035000 tx=0x7f8710030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.030+0000 7f871ea00640 1 -- 172.21.15.150:0/4058452718 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8718059ea0 con 0x7f87280ce850 2024-04-15T15:25:46.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.231+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f872815be50 con 0x7f87280ce850 2024-04-15T15:25:46.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.232+0000 7f871ea00640 1 -- 172.21.15.150:0/4058452718 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f87180597a0 con 0x7f87280ce850 2024-04-15T15:25:46.234 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:25:46.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.234+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f870406c980 msgr2=0x7f870406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.234+0000 7f8727e00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f870406c980 0x7f870406ee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8710035000 tx=0x7f8710030000 comp rx=0 tx=0).stop 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.234+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87280ce850 msgr2=0x7f87280aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.234+0000 7f8727e00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87280ce850 0x7f87280aee70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f871800c430 tx=0x7f871800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.235+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 shutdown_connections 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.235+0000 7f8727e00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8728146e10 0x7f87280b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.235+0000 7f8727e00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f870406c980 0x7f870406ee40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.235+0000 7f8727e00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872815d510 0x7f87281e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.235+0000 7f8727e00640 1 --2- 172.21.15.150:0/4058452718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87280ce850 0x7f87280aee70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.235+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 >> 172.21.15.150:0/4058452718 conn(0x7f87280b23a0 msgr2=0x7f872815a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.235+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 shutdown_connections 2024-04-15T15:25:46.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.235+0000 7f8727e00640 1 -- 172.21.15.150:0/4058452718 wait complete. 2024-04-15T15:25:46.255 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:25:46.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.472+0000 7ff802400640 1 Processor -- start 2024-04-15T15:25:46.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.472+0000 7ff802400640 1 -- start start 2024-04-15T15:25:46.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.473+0000 7ff802400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc1461c0 0x7ff7fc071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.473+0000 7ff802400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7fc0cebb0 0x7ff7fc06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.473+0000 7ff802400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7fc0cd2d0 con 0x7ff7fc072420 2024-04-15T15:25:46.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.473+0000 7ff802400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7ff7fc0cf1d0 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.473+0000 7ff802400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7fc072880 con 0x7ff7fc1461c0 2024-04-15T15:25:46.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff801a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7fc0cebb0 0x7ff7fc06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff7f3e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7ff7fc072420 0x7ff7fc0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:47744/0 (socket says 172.21.15.150:47744) 2024-04-15T15:25:46.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff801a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7fc0cebb0 0x7ff7fc06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36346/0 (socket says 172.21.15.150:36346) 2024-04-15T15:25:46.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff7f3e00640 1 -- 172.21.15.150:0/3674465195 learned_addr learned my addr 172.21.15.150:0/3674465195 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:46.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff801000640 1 --2- 172.21.15.150:0/3674465195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc1461c0 0x7ff7fc071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff801a00640 1 -- 172.21.15.150:0/3674465195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc1461c0 msgr2=0x7ff7fc071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:46.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff801a00640 1 --2- 172.21.15.150:0/3674465195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc1461c0 0x7ff7fc071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff801a00640 1 -- 172.21.15.150:0/3674465195 >> v1:172.21.15.150:6789/0 conn(0x7ff7fc072420 legacy=0x7ff7fc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.474+0000 7ff801a00640 1 -- 172.21.15.150:0/3674465195 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc0cd5b0 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.476+0000 7ff801a00640 1 --2- 172.21.15.150:0/3674465195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc06df20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009d90 tx=0x7ff7f402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37dba0d38907c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.476+0000 7ff7fbe00640 1 -- 172.21.15.150:0/3674465195 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4039070 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.476+0000 7ff7fbe00640 1 -- 172.21.15.150:0/3674465195 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f402cc60 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.476+0000 7ff7fbe00640 1 -- 172.21.15.150:0/3674465195 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4034660 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.477+0000 7ff802400640 1 -- 172.21.15.150:0/3674465195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 msgr2=0x7ff7fc06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.477+0000 7ff802400640 1 --2- 172.21.15.150:0/3674465195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc06df20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009d90 tx=0x7ff7f402beb0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.477+0000 7ff802400640 1 -- 172.21.15.150:0/3674465195 shutdown_connections 2024-04-15T15:25:46.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.477+0000 7ff802400640 1 --2- 172.21.15.150:0/3674465195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc06df20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.477+0000 7ff802400640 1 --2- 172.21.15.150:0/3674465195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc1461c0 0x7ff7fc071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.477+0000 7ff802400640 1 -- 172.21.15.150:0/3674465195 >> 172.21.15.150:0/3674465195 conn(0x7ff7fc06b930 msgr2=0x7ff7fc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:46.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.477+0000 7ff802400640 1 -- 172.21.15.150:0/3674465195 shutdown_connections 2024-04-15T15:25:46.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.477+0000 7ff802400640 1 -- 172.21.15.150:0/3674465195 wait complete. 2024-04-15T15:25:46.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.479+0000 7ff802400640 1 Processor -- start 2024-04-15T15:25:46.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.479+0000 7ff802400640 1 -- start start 2024-04-15T15:25:46.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff802400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc072420 0x7ff7fc0ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff802400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff802400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7fc1461c0 0x7ff7fc0b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff802400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc0cdb70 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff802400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc0cd080 con 0x7ff7fc1461c0 2024-04-15T15:25:46.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff802400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7fc06e460 con 0x7ff7fc072420 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff7f3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff7f3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36358/0 (socket says 172.21.15.150:36358) 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff7f3e00640 1 -- 172.21.15.150:0/586635290 learned_addr learned my addr 172.21.15.150:0/586635290 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff801000640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc072420 0x7ff7fc0ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.481+0000 7ff801a00640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7fc1461c0 0x7ff7fc0b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f3e00640 1 -- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc072420 msgr2=0x7ff7fc0ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f3e00640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc072420 0x7ff7fc0ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f3e00640 1 -- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7fc1461c0 msgr2=0x7ff7fc0b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f3e00640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7fc1461c0 0x7ff7fc0b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f3e00640 1 -- 172.21.15.150:0/586635290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc1f8c70 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f3e00640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc156830 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c400 tx=0x7ff7e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f9600640 1 -- 172.21.15.150:0/586635290 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4017070 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f9600640 1 -- 172.21.15.150:0/586635290 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e4005ce0 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff7f9600640 1 -- 172.21.15.150:0/586635290 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40072f0 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.482+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7fc0ab3f0 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.483+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7fc0abc30 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.484+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7fc0755c0 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.485+0000 7ff7f9600640 1 -- 172.21.15.150:0/586635290 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff7e401a070 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.486+0000 7ff7f9600640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d806c980 0x7ff7d806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.486+0000 7ff7f9600640 1 -- 172.21.15.150:0/586635290 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7ff7e408b070 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.487+0000 7ff801000640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d806c980 0x7ff7d806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.488+0000 7ff801000640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d806c980 0x7ff7d806ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec006fd0 tx=0x7ff7ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.491+0000 7ff7f9600640 1 -- 172.21.15.150:0/586635290 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7fc0ab3f0 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.688+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff7fc072e40 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.689+0000 7ff7f9600640 1 -- 172.21.15.150:0/586635290 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7ff7e40593e0 con 0x7ff7fc0cebb0 2024-04-15T15:25:46.690 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d806c980 msgr2=0x7ff7d806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d806c980 0x7ff7d806ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec006fd0 tx=0x7ff7ec008040 comp rx=0 tx=0).stop 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 msgr2=0x7ff7fc156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc156830 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c400 tx=0x7ff7e400c8c0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 shutdown_connections 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7fc1461c0 0x7ff7fc0b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7fc0cebb0 0x7ff7fc156830 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff7d806c980 0x7ff7d806ee40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 --2- 172.21.15.150:0/586635290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7fc072420 0x7ff7fc0ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 >> 172.21.15.150:0/586635290 conn(0x7ff7fc06b930 msgr2=0x7ff7fc15b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:46.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 shutdown_connections 2024-04-15T15:25:46.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.691+0000 7ff802400640 1 -- 172.21.15.150:0/586635290 wait complete. 2024-04-15T15:25:46.711 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:25:46.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.910+0000 7f8c9e400640 1 Processor -- start 2024-04-15T15:25:46.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.910+0000 7f8c9e400640 1 -- start start 2024-04-15T15:25:46.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c9e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980ce650 0x7f8c9809ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c9e400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8c980d17c0 0x7f8c9809b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c9e400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c98071330 con 0x7f8c980703d0 2024-04-15T15:25:46.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c9e400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f8c9809bb40 con 0x7f8c980d17c0 2024-04-15T15:25:46.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c9e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9806ce50 con 0x7f8c980ce650 2024-04-15T15:25:46.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c97e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8c980d17c0 0x7f8c9809b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c96a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f8c980703d0 0x7f8c9806d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:53664/0 (socket says 172.21.15.150:53664) 2024-04-15T15:25:46.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c97e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8c980d17c0 0x7f8c9809b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49478/0 (socket says 172.21.15.150:49478) 2024-04-15T15:25:46.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c96a00640 1 -- 172.21.15.150:0/3129070210 learned_addr learned my addr 172.21.15.150:0/3129070210 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:46.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c97e00640 1 -- 172.21.15.150:0/3129070210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980ce650 msgr2=0x7f8c9809ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:46.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c97e00640 1 --2- 172.21.15.150:0/3129070210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980ce650 0x7f8c9809ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c97e00640 1 -- 172.21.15.150:0/3129070210 >> v1:172.21.15.150:6789/0 conn(0x7f8c980703d0 legacy=0x7f8c9806d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.912+0000 7f8c97e00640 1 -- 172.21.15.150:0/3129070210 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9806cae0 con 0x7f8c980d17c0 2024-04-15T15:25:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.913+0000 7f8c97e00640 1 --2- 172.21.15.150:0/3129070210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980d17c0 0x7f8c9809b060 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009d90 tx=0x7f8c9002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e75859f719667 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c96000640 1 -- 172.21.15.150:0/3129070210 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90039070 con 0x7f8c980d17c0 2024-04-15T15:25:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c96000640 1 -- 172.21.15.150:0/3129070210 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9002cc60 con 0x7f8c980d17c0 2024-04-15T15:25:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c96000640 1 -- 172.21.15.150:0/3129070210 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90034660 con 0x7f8c980d17c0 2024-04-15T15:25:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c9e400640 1 -- 172.21.15.150:0/3129070210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980d17c0 msgr2=0x7f8c9809b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c9e400640 1 --2- 172.21.15.150:0/3129070210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980d17c0 0x7f8c9809b060 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009d90 tx=0x7f8c9002beb0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c9e400640 1 -- 172.21.15.150:0/3129070210 shutdown_connections 2024-04-15T15:25:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c9e400640 1 --2- 172.21.15.150:0/3129070210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980d17c0 0x7f8c9809b060 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c9e400640 1 --2- 172.21.15.150:0/3129070210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980ce650 0x7f8c9809ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c9e400640 1 -- 172.21.15.150:0/3129070210 >> 172.21.15.150:0/3129070210 conn(0x7f8c980d2720 msgr2=0x7f8c980d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c9e400640 1 -- 172.21.15.150:0/3129070210 shutdown_connections 2024-04-15T15:25:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.914+0000 7f8c9e400640 1 -- 172.21.15.150:0/3129070210 wait complete. 2024-04-15T15:25:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.916+0000 7f8c9e400640 1 Processor -- start 2024-04-15T15:25:46.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.916+0000 7f8c9e400640 1 -- start start 2024-04-15T15:25:46.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.917+0000 7f8c9e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c980703d0 0x7f8c980cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.917+0000 7f8c9e400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980ce650 0x7f8c98152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.917+0000 7f8c9e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980d17c0 0x7f8c980cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.917+0000 7f8c9e400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9809bd90 con 0x7f8c980ce650 2024-04-15T15:25:46.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.917+0000 7f8c9e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98071610 con 0x7f8c980703d0 2024-04-15T15:25:46.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.917+0000 7f8c9e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8c980cea30 con 0x7f8c980d17c0 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.917+0000 7f8c96a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980ce650 0x7f8c98152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.917+0000 7f8c96a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980ce650 0x7f8c98152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49484/0 (socket says 172.21.15.150:49484) 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c96a00640 1 -- 172.21.15.150:0/2428087647 learned_addr learned my addr 172.21.15.150:0/2428087647 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c97e00640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980d17c0 0x7f8c980cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c97400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c980703d0 0x7f8c980cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c96a00640 1 -- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980d17c0 msgr2=0x7f8c980cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c96a00640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980d17c0 0x7f8c980cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c96a00640 1 -- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c980703d0 msgr2=0x7f8c980cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c96a00640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c980703d0 0x7f8c980cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c96a00640 1 -- 172.21.15.150:0/2428087647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c98153100 con 0x7f8c980ce650 2024-04-15T15:25:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c96a00640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980ce650 0x7f8c98152bc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800c440 tx=0x7f8c88001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.918+0000 7f8c8f400640 1 -- 172.21.15.150:0/2428087647 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88017070 con 0x7f8c980ce650 2024-04-15T15:25:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.919+0000 7f8c8f400640 1 -- 172.21.15.150:0/2428087647 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c88005ce0 con 0x7f8c980ce650 2024-04-15T15:25:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.919+0000 7f8c8f400640 1 -- 172.21.15.150:0/2428087647 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c880072f0 con 0x7f8c980ce650 2024-04-15T15:25:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.919+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c980c9db0 con 0x7f8c980ce650 2024-04-15T15:25:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.919+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c980ca370 con 0x7f8c980ce650 2024-04-15T15:25:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.920+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c980b8550 con 0x7f8c980ce650 2024-04-15T15:25:46.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.921+0000 7f8c8f400640 1 -- 172.21.15.150:0/2428087647 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8c8801a070 con 0x7f8c980ce650 2024-04-15T15:25:46.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.922+0000 7f8c8f400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8c7806c980 0x7f8c7806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:46.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.922+0000 7f8c8f400640 1 -- 172.21.15.150:0/2428087647 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f8c8808b0d0 con 0x7f8c980ce650 2024-04-15T15:25:46.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.922+0000 7f8c97400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8c7806c980 0x7f8c7806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.924+0000 7f8c97400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8c7806c980 0x7f8c7806ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009890 tx=0x7f8c84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:46.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:46.927+0000 7f8c8f400640 1 -- 172.21.15.150:0/2428087647 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c980b8550 con 0x7f8c980ce650 2024-04-15T15:25:46.964 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:25:46.970 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:25:47.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.136+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8c980d1d00 con 0x7f8c980ce650 2024-04-15T15:25:47.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.137+0000 7f8c8f400640 1 -- 172.21.15.150:0/2428087647 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f8c880593b0 con 0x7f8c980ce650 2024-04-15T15:25:47.139 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8c7806c980 msgr2=0x7f8c7806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8c7806c980 0x7f8c7806ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009890 tx=0x7f8c84008040 comp rx=0 tx=0).stop 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980ce650 msgr2=0x7f8c98152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980ce650 0x7f8c98152bc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800c440 tx=0x7f8c88001e70 comp rx=0 tx=0).stop 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 shutdown_connections 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8c7806c980 0x7f8c7806ee40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8c980d17c0 0x7f8c980cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c980ce650 0x7f8c98152bc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 --2- 172.21.15.150:0/2428087647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c980703d0 0x7f8c980cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 >> 172.21.15.150:0/2428087647 conn(0x7f8c980d2720 msgr2=0x7f8c980b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:47.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 shutdown_connections 2024-04-15T15:25:47.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.140+0000 7f8c9e400640 1 -- 172.21.15.150:0/2428087647 wait complete. 2024-04-15T15:25:47.162 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:25:47.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.371+0000 7f6e16800640 1 Processor -- start 2024-04-15T15:25:47.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.371+0000 7f6e16800640 1 -- start start 2024-04-15T15:25:47.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.372+0000 7f6e16800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6e1015d770 0x7f6e100cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:47.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.372+0000 7f6e16800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6e100b59b0 0x7f6e100b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:47.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e16800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e100cd940 con 0x7f6e100b0cb0 2024-04-15T15:25:47.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e16800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f6e100ce550 con 0x7f6e1015d770 2024-04-15T15:25:47.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e16800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6e100cdc20 con 0x7f6e100b59b0 2024-04-15T15:25:47.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e15400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6e100b0cb0 0x7f6e100ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:53680/0 (socket says 172.21.15.150:53680) 2024-04-15T15:25:47.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e15400640 1 -- 172.21.15.150:0/3140506448 learned_addr learned my addr 172.21.15.150:0/3140506448 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e14a00640 1 --2- 172.21.15.150:0/3140506448 >> v2:172.21.15.150:3300/0 conn(0x7f6e1015d770 0x7f6e100cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e0fe00640 1 -- 172.21.15.150:0/3140506448 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 684240554 0 0) 0x7f6e100cd940 con 0x7f6e100b0cb0 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e0fe00640 1 -- 172.21.15.150:0/3140506448 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df8003520 con 0x7f6e100b0cb0 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e15e00640 1 --2- 172.21.15.150:0/3140506448 >> v2:172.21.15.187:3300/0 conn(0x7f6e100b59b0 0x7f6e100b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e14a00640 1 -- 172.21.15.150:0/3140506448 >> v2:172.21.15.187:3300/0 conn(0x7f6e100b59b0 msgr2=0x7f6e100b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e14a00640 1 --2- 172.21.15.150:0/3140506448 >> v2:172.21.15.187:3300/0 conn(0x7f6e100b59b0 0x7f6e100b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e14a00640 1 -- 172.21.15.150:0/3140506448 >> v1:172.21.15.150:6789/0 conn(0x7f6e100b0cb0 legacy=0x7f6e100ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.373+0000 7f6e14a00640 1 -- 172.21.15.150:0/3140506448 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100cdf00 con 0x7f6e1015d770 2024-04-15T15:25:47.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.375+0000 7f6e14a00640 1 --2- 172.21.15.150:0/3140506448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1015d770 0x7f6e100cd020 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6e100b3410 tx=0x7f6dfc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93f41704ac7f52b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:47.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.375+0000 7f6e0fe00640 1 -- 172.21.15.150:0/3140506448 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc035020 con 0x7f6e1015d770 2024-04-15T15:25:47.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.375+0000 7f6e0fe00640 1 -- 172.21.15.150:0/3140506448 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc031830 con 0x7f6e1015d770 2024-04-15T15:25:47.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.375+0000 7f6e0fe00640 1 -- 172.21.15.150:0/3140506448 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc031be0 con 0x7f6e1015d770 2024-04-15T15:25:47.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.376+0000 7f6e16800640 1 -- 172.21.15.150:0/3140506448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1015d770 msgr2=0x7f6e100cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:47.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.376+0000 7f6e16800640 1 --2- 172.21.15.150:0/3140506448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1015d770 0x7f6e100cd020 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6e100b3410 tx=0x7f6dfc02b8d0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.376+0000 7f6e16800640 1 -- 172.21.15.150:0/3140506448 shutdown_connections 2024-04-15T15:25:47.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.376+0000 7f6e16800640 1 --2- 172.21.15.150:0/3140506448 >> v2:172.21.15.187:3300/0 conn(0x7f6e100b59b0 0x7f6e100b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.376+0000 7f6e16800640 1 --2- 172.21.15.150:0/3140506448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e1015d770 0x7f6e100cd020 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.376+0000 7f6e16800640 1 -- 172.21.15.150:0/3140506448 >> 172.21.15.150:0/3140506448 conn(0x7f6e100b23a0 msgr2=0x7f6e100b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:47.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.376+0000 7f6e16800640 1 -- 172.21.15.150:0/3140506448 shutdown_connections 2024-04-15T15:25:47.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.376+0000 7f6e16800640 1 -- 172.21.15.150:0/3140506448 wait complete. 2024-04-15T15:25:47.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.378+0000 7f6e16800640 1 Processor -- start 2024-04-15T15:25:47.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.378+0000 7f6e16800640 1 -- start start 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e16800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e100b0cb0 0x7f6e100b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e16800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e100b59b0 0x7f6e10148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e16800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e1015d770 0x7f6e10154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e16800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100cd560 con 0x7f6e100b0cb0 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e16800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100b1090 con 0x7f6e1015d770 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e16800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6e100cd6e0 con 0x7f6e100b59b0 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e15400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e100b0cb0 0x7f6e100b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e15400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e100b0cb0 0x7f6e100b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49512/0 (socket says 172.21.15.150:49512) 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e15400640 1 -- 172.21.15.150:0/4263632773 learned_addr learned my addr 172.21.15.150:0/4263632773 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:47.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e15400640 1 -- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e100b59b0 msgr2=0x7f6e10148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e15400640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e100b59b0 0x7f6e10148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.379+0000 7f6e15400640 1 -- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e1015d770 msgr2=0x7f6e10154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.380+0000 7f6e15400640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e1015d770 0x7f6e10154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.380+0000 7f6e15400640 1 -- 172.21.15.150:0/4263632773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100b4c60 con 0x7f6e100b0cb0 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.380+0000 7f6e15400640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e100b0cb0 0x7f6e100b4430 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c390 tx=0x7f6e0400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.380+0000 7f6e0d600640 1 -- 172.21.15.150:0/4263632773 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04017070 con 0x7f6e100b0cb0 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.380+0000 7f6e0d600640 1 -- 172.21.15.150:0/4263632773 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e04005ce0 con 0x7f6e100b0cb0 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.380+0000 7f6e0d600640 1 -- 172.21.15.150:0/4263632773 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040072f0 con 0x7f6e100b0cb0 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.380+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e100b4f40 con 0x7f6e100b0cb0 2024-04-15T15:25:47.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.380+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e100b4970 con 0x7f6e100b0cb0 2024-04-15T15:25:47.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.382+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e100755c0 con 0x7f6e100b0cb0 2024-04-15T15:25:47.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.382+0000 7f6e0d600640 1 -- 172.21.15.150:0/4263632773 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6e0401a070 con 0x7f6e100b0cb0 2024-04-15T15:25:47.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.384+0000 7f6e0d600640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6df006c980 0x7f6df006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:47.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.384+0000 7f6e0d600640 1 -- 172.21.15.150:0/4263632773 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f6e0408af90 con 0x7f6e100b0cb0 2024-04-15T15:25:47.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.384+0000 7f6e14a00640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6df006c980 0x7f6df006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:47.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.386+0000 7f6e14a00640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6df006c980 0x7f6df006ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc036000 tx=0x7f6dfc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:47.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.388+0000 7f6e0d600640 1 -- 172.21.15.150:0/4263632773 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e04059400 con 0x7f6e100b0cb0 2024-04-15T15:25:47.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.601+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e100b5460 con 0x7f6e100b0cb0 2024-04-15T15:25:47.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.602+0000 7f6e0d600640 1 -- 172.21.15.150:0/4263632773 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+124 (secure 0 0 0) 0x7f6e04058d00 con 0x7f6e100b0cb0 2024-04-15T15:25:47.604 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6835/3327940880 2024-04-16T15:25:33.990360+0000 2024-04-15T15:25:47.604 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6833/3327940880 2024-04-16T15:25:33.990360+0000 2024-04-15T15:25:47.604 INFO:teuthology.orchestra.run.smithi150.stderr:listed 2 entries 2024-04-15T15:25:47.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6df006c980 msgr2=0x7f6df006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6df006c980 0x7f6df006ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc036000 tx=0x7f6dfc030000 comp rx=0 tx=0).stop 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e100b0cb0 msgr2=0x7f6e100b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e100b0cb0 0x7f6e100b4430 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c390 tx=0x7f6e0400c850 comp rx=0 tx=0).stop 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 shutdown_connections 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e1015d770 0x7f6e10154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6df006c980 0x7f6df006ee40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e100b59b0 0x7f6e10148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 --2- 172.21.15.150:0/4263632773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e100b0cb0 0x7f6e100b4430 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 >> 172.21.15.150:0/4263632773 conn(0x7f6e100b23a0 msgr2=0x7f6e100ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 shutdown_connections 2024-04-15T15:25:47.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:47.605+0000 7f6e16800640 1 -- 172.21.15.150:0/4263632773 wait complete. 2024-04-15T15:25:47.625 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-15T15:25:47.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:47.625 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.2 2024-04-15T15:25:47.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:25:47.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:47.725 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-15T15:25:47.775 INFO:tasks.cephfs.mount:Terminating background process 2024-04-15T15:25:47.775 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:25:47.775 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:47.832 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:25:47.832 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:25:47.881 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:48.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.085+0000 7f649be00640 1 Processor -- start 2024-04-15T15:25:48.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.085+0000 7f649be00640 1 -- start start 2024-04-15T15:25:48.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.087+0000 7f649be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c162380 0x7f649c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.087+0000 7f649be00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649c162940 con 0x7f649c148130 2024-04-15T15:25:48.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.087+0000 7f649be00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649c162c20 con 0x7f649c072660 2024-04-15T15:25:48.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.087+0000 7f649be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f649c148510 con 0x7f649c162380 2024-04-15T15:25:48.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.087+0000 7f649aa00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f649c148130 0x7f649c071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:53684/0 (socket says 172.21.15.150:53684) 2024-04-15T15:25:48.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.087+0000 7f649aa00640 1 -- 172.21.15.150:0/3807347955 learned_addr learned my addr 172.21.15.150:0/3807347955 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:48.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.087+0000 7f649b400640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c162380 0x7f649c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242410725 0 0) 0x7f649c162940 con 0x7f649c148130 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6480003520 con 0x7f649c148130 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3941405137 0 0) 0x7f6480003520 con 0x7f649c148130 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c162380 msgr2=0x7f649c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c162380 0x7f649c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 >> v1:172.21.15.187:6789/0 conn(0x7f649c072660 legacy=0x7f649c0af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c162f00 con 0x7f649c148130 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f648c001670 con 0x7f649c148130 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 >> v1:172.21.15.150:6789/0 conn(0x7f649c148130 legacy=0x7f649c071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64800039e0 0x7f6480003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6480004800 0x7f6480004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c162380 0x7f64800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f649c148510 con 0x7f64800039e0 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f648000ba70 con 0x7f6480004800 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f648000bc70 con 0x7f649c162380 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.088+0000 7f649aa00640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64800039e0 0x7f6480003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.089+0000 7f649a000640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6480004800 0x7f6480004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.089+0000 7f649b400640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c162380 0x7f64800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.089+0000 7f649aa00640 1 -- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c162380 msgr2=0x7f64800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.089+0000 7f649aa00640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c162380 0x7f64800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.089+0000 7f649aa00640 1 -- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6480004800 msgr2=0x7f6480004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.089+0000 7f649aa00640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6480004800 0x7f6480004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.089+0000 7f649aa00640 1 -- 172.21.15.150:0/3807347955 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c162f00 con 0x7f64800039e0 2024-04-15T15:25:48.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.089+0000 7f649a000640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6480004800 0x7f6480004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:25:48.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.090+0000 7f649aa00640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64800039e0 0x7f6480003e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f648c001050 tx=0x7f6480005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeebe0b1442c46c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.091+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c007dd0 con 0x7f64800039e0 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.091+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c030070 con 0x7f64800039e0 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.091+0000 7f6499600640 1 -- 172.21.15.150:0/3807347955 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c02cdd0 con 0x7f64800039e0 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.091+0000 7f649be00640 1 -- 172.21.15.150:0/3807347955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64800039e0 msgr2=0x7f6480003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.091+0000 7f649be00640 1 --2- 172.21.15.150:0/3807347955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64800039e0 0x7f6480003e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f648c001050 tx=0x7f6480005830 comp rx=0 tx=0).stop 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.091+0000 7f649b400640 1 -- 172.21.15.150:0/3807347955 reap_dead start 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.092+0000 7f649be00640 1 -- 172.21.15.150:0/3807347955 shutdown_connections 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.092+0000 7f649be00640 1 -- 172.21.15.150:0/3807347955 >> 172.21.15.150:0/3807347955 conn(0x7f649c06b930 msgr2=0x7f649c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.092+0000 7f649be00640 1 -- 172.21.15.150:0/3807347955 shutdown_connections 2024-04-15T15:25:48.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.092+0000 7f649be00640 1 -- 172.21.15.150:0/3807347955 wait complete. 2024-04-15T15:25:48.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.094+0000 7f649be00640 1 Processor -- start 2024-04-15T15:25:48.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.094+0000 7f649be00640 1 -- start start 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64800039e0 0x7f649c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c0ce090 0x7f649c1f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c15c690 0x7f649c146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f649c0aed10 con 0x7f649c15c690 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f649c0aeb90 con 0x7f64800039e0 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f649c15ff50 con 0x7f649c0ce090 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c15c690 0x7f649c146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c15c690 0x7f649c146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49532/0 (socket says 172.21.15.150:49532) 2024-04-15T15:25:48.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649b400640 1 -- 172.21.15.150:0/4106624472 learned_addr learned my addr 172.21.15.150:0/4106624472 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:48.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649b400640 1 -- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c0ce090 msgr2=0x7f649c1f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649b400640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c0ce090 0x7f649c1f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649b400640 1 -- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64800039e0 msgr2=0x7f649c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649b400640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64800039e0 0x7f649c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.095+0000 7f649b400640 1 -- 172.21.15.150:0/4106624472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c0ce5e0 con 0x7f649c15c690 2024-04-15T15:25:48.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.096+0000 7f649b400640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c15c690 0x7f649c146af0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f649c0ae620 tx=0x7f64940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:48.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.096+0000 7f6492a00640 1 -- 172.21.15.150:0/4106624472 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494011e70 con 0x7f649c15c690 2024-04-15T15:25:48.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.096+0000 7f6492a00640 1 -- 172.21.15.150:0/4106624472 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649401b070 con 0x7f649c15c690 2024-04-15T15:25:48.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.096+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c0ce8c0 con 0x7f649c15c690 2024-04-15T15:25:48.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.096+0000 7f6492a00640 1 -- 172.21.15.150:0/4106624472 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494003990 con 0x7f649c15c690 2024-04-15T15:25:48.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.096+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c0cee80 con 0x7f649c15c690 2024-04-15T15:25:48.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.097+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649c0755c0 con 0x7f649c15c690 2024-04-15T15:25:48.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.098+0000 7f6492a00640 1 -- 172.21.15.150:0/4106624472 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6494005d10 con 0x7f649c15c690 2024-04-15T15:25:48.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.099+0000 7f6492a00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f647c06c9b0 0x7f647c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.099+0000 7f6492a00640 1 -- 172.21.15.150:0/4106624472 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f649408c170 con 0x7f649c15c690 2024-04-15T15:25:48.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.100+0000 7f649aa00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f647c06c9b0 0x7f647c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.101+0000 7f649aa00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f647c06c9b0 0x7f647c06ee70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f648c034000 tx=0x7f648c039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:48.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.103+0000 7f6492a00640 1 -- 172.21.15.150:0/4106624472 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f649401d310 con 0x7f649c15c690 2024-04-15T15:25:48.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.305+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f649c06f6d0 con 0x7f649c15c690 2024-04-15T15:25:48.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.308+0000 7f6492a00640 1 -- 172.21.15.150:0/4106624472 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16848 (secure 0 0 0) 0x7f649405a340 con 0x7f649c15c690 2024-04-15T15:25:48.311 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:48.311 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":36,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:33.990388+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/3327940880":"2024-04-16T15:25:33.990360+0000","172.21.15.150:6833/3327940880":"2024-04-16T15:25:33.990360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:48.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.311+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f647c06c9b0 msgr2=0x7f647c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.311+0000 7f649be00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f647c06c9b0 0x7f647c06ee70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f648c034000 tx=0x7f648c039040 comp rx=0 tx=0).stop 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.311+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c15c690 msgr2=0x7f649c146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.311+0000 7f649be00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c15c690 0x7f649c146af0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f649c0ae620 tx=0x7f64940115a0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.312+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 shutdown_connections 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.312+0000 7f649be00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c15c690 0x7f649c146af0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.312+0000 7f649be00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f647c06c9b0 0x7f647c06ee70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.312+0000 7f649be00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f649c0ce090 0x7f649c1f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.312+0000 7f649be00640 1 --2- 172.21.15.150:0/4106624472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64800039e0 0x7f649c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.312+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 >> 172.21.15.150:0/4106624472 conn(0x7f649c06b930 msgr2=0x7f649c142f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.312+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 shutdown_connections 2024-04-15T15:25:48.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.312+0000 7f649be00640 1 -- 172.21.15.150:0/4106624472 wait complete. 2024-04-15T15:25:48.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:25:04.190900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:25:05.324314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:25:48.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:25:48.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.543+0000 7f26eb200640 1 Processor -- start 2024-04-15T15:25:48.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.543+0000 7f26eb200640 1 -- start start 2024-04-15T15:25:48.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.544+0000 7f26eb200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f26ec147fc0 0x7f26ec0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.544+0000 7f26eb200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f26ec162b00 0x7f26ec06df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.545+0000 7f26eb200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26ec0b5e00 con 0x7f26ec0aef00 2024-04-15T15:25:48.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.545+0000 7f26eb200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f26ec1483a0 con 0x7f26ec147fc0 2024-04-15T15:25:48.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.545+0000 7f26eb200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f26ec0afaa0 con 0x7f26ec162b00 2024-04-15T15:25:48.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.545+0000 7f26e1400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f26ec0aef00 0x7f26ec1623f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:53688/0 (socket says 172.21.15.150:53688) 2024-04-15T15:25:48.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.545+0000 7f26e1400640 1 -- 172.21.15.150:0/2047073357 learned_addr learned my addr 172.21.15.150:0/2047073357 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:48.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.545+0000 7f26e9e00640 1 --2- 172.21.15.150:0/2047073357 >> v2:172.21.15.150:3300/0 conn(0x7f26ec147fc0 0x7f26ec0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.545+0000 7f26ea800640 1 --2- 172.21.15.150:0/2047073357 >> v2:172.21.15.187:3300/0 conn(0x7f26ec162b00 0x7f26ec06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.546+0000 7f26e9400640 1 -- 172.21.15.150:0/2047073357 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073922801 0 0) 0x7f26ec0b5e00 con 0x7f26ec0aef00 2024-04-15T15:25:48.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.546+0000 7f26e9400640 1 -- 172.21.15.150:0/2047073357 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26d0003520 con 0x7f26ec0aef00 2024-04-15T15:25:48.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.546+0000 7f26e9e00640 1 -- 172.21.15.150:0/2047073357 >> v2:172.21.15.187:3300/0 conn(0x7f26ec162b00 msgr2=0x7f26ec06df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:48.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.546+0000 7f26e9e00640 1 --2- 172.21.15.150:0/2047073357 >> v2:172.21.15.187:3300/0 conn(0x7f26ec162b00 0x7f26ec06df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.546+0000 7f26e9e00640 1 -- 172.21.15.150:0/2047073357 >> v1:172.21.15.150:6789/0 conn(0x7f26ec0aef00 legacy=0x7f26ec1623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.546+0000 7f26e9e00640 1 -- 172.21.15.150:0/2047073357 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec0afd80 con 0x7f26ec147fc0 2024-04-15T15:25:48.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.546+0000 7f26ea800640 1 --2- 172.21.15.150:0/2047073357 >> v2:172.21.15.187:3300/0 conn(0x7f26ec162b00 0x7f26ec06df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.547+0000 7f26e9e00640 1 --2- 172.21.15.150:0/2047073357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec147fc0 0x7f26ec0b36a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f26ec06fef0 tx=0x7f26dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bd70a4e567b711 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.547+0000 7f26e9400640 1 -- 172.21.15.150:0/2047073357 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0315a0 con 0x7f26ec147fc0 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26e9400640 1 -- 172.21.15.150:0/2047073357 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26dc037070 con 0x7f26ec147fc0 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26e9400640 1 -- 172.21.15.150:0/2047073357 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc031960 con 0x7f26ec147fc0 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26eb200640 1 -- 172.21.15.150:0/2047073357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec147fc0 msgr2=0x7f26ec0b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26eb200640 1 --2- 172.21.15.150:0/2047073357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec147fc0 0x7f26ec0b36a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f26ec06fef0 tx=0x7f26dc02b840 comp rx=0 tx=0).stop 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26eb200640 1 -- 172.21.15.150:0/2047073357 shutdown_connections 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26eb200640 1 --2- 172.21.15.150:0/2047073357 >> v2:172.21.15.187:3300/0 conn(0x7f26ec162b00 0x7f26ec06df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26eb200640 1 --2- 172.21.15.150:0/2047073357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec147fc0 0x7f26ec0b36a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26eb200640 1 -- 172.21.15.150:0/2047073357 >> 172.21.15.150:0/2047073357 conn(0x7f26ec0b23a0 msgr2=0x7f26ec0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26eb200640 1 -- 172.21.15.150:0/2047073357 shutdown_connections 2024-04-15T15:25:48.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.548+0000 7f26eb200640 1 -- 172.21.15.150:0/2047073357 wait complete. 2024-04-15T15:25:48.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.550+0000 7f26eb200640 1 Processor -- start 2024-04-15T15:25:48.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.551+0000 7f26eb200640 1 -- start start 2024-04-15T15:25:48.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26eb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26ec0aef00 0x7f26ec15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26eb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26ec147fc0 0x7f26ec141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26eb200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec162b00 0x7f26ec15fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26eb200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec0b3be0 con 0x7f26ec162b00 2024-04-15T15:25:48.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26eb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec0b0060 con 0x7f26ec147fc0 2024-04-15T15:25:48.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26eb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f26ec0b3d60 con 0x7f26ec0aef00 2024-04-15T15:25:48.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26ea800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec162b00 0x7f26ec15fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26ea800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec162b00 0x7f26ec15fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49562/0 (socket says 172.21.15.150:49562) 2024-04-15T15:25:48.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26ea800640 1 -- 172.21.15.150:0/1920663786 learned_addr learned my addr 172.21.15.150:0/1920663786 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:48.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.552+0000 7f26e1400640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26ec147fc0 0x7f26ec141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26e9e00640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26ec0aef00 0x7f26ec15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26ea800640 1 -- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26ec0aef00 msgr2=0x7f26ec15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26ea800640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26ec0aef00 0x7f26ec15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26ea800640 1 -- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26ec147fc0 msgr2=0x7f26ec141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26ea800640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26ec147fc0 0x7f26ec141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26ea800640 1 -- 172.21.15.150:0/1920663786 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec142bc0 con 0x7f26ec162b00 2024-04-15T15:25:48.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26e1400640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26ec147fc0 0x7f26ec141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:48.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26e9e00640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26ec0aef00 0x7f26ec15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:48.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26ea800640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec162b00 0x7f26ec15fae0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f26e4007df0 tx=0x7f26e400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:48.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26e2a00640 1 -- 172.21.15.150:0/1920663786 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e401a070 con 0x7f26ec162b00 2024-04-15T15:25:48.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26e2a00640 1 -- 172.21.15.150:0/1920663786 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e4003990 con 0x7f26ec162b00 2024-04-15T15:25:48.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26e2a00640 1 -- 172.21.15.150:0/1920663786 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4003d80 con 0x7f26ec162b00 2024-04-15T15:25:48.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ec142ea0 con 0x7f26ec162b00 2024-04-15T15:25:48.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.553+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ec143460 con 0x7f26ec162b00 2024-04-15T15:25:48.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.555+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ec0755c0 con 0x7f26ec162b00 2024-04-15T15:25:48.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.556+0000 7f26e2a00640 1 -- 172.21.15.150:0/1920663786 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f26e401d070 con 0x7f26ec162b00 2024-04-15T15:25:48.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.557+0000 7f26e2a00640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f26c806c980 0x7f26c806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:48.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.557+0000 7f26e2a00640 1 -- 172.21.15.150:0/1920663786 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f26e408c090 con 0x7f26ec162b00 2024-04-15T15:25:48.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.558+0000 7f26e9e00640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f26c806c980 0x7f26c806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:48.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.559+0000 7f26e9e00640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f26c806c980 0x7f26c806ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f26dc036000 tx=0x7f26dc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:48.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.562+0000 7f26e2a00640 1 -- 172.21.15.150:0/1920663786 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26e4010070 con 0x7f26ec162b00 2024-04-15T15:25:48.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.761+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26ec143660 con 0x7f26ec162b00 2024-04-15T15:25:48.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.762+0000 7f26e2a00640 1 -- 172.21.15.150:0/1920663786 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f26e405a260 con 0x7f26ec162b00 2024-04-15T15:25:48.763 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:25:48.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f26c806c980 msgr2=0x7f26c806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f26c806c980 0x7f26c806ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f26dc036000 tx=0x7f26dc008030 comp rx=0 tx=0).stop 2024-04-15T15:25:48.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec162b00 msgr2=0x7f26ec15fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec162b00 0x7f26ec15fae0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f26e4007df0 tx=0x7f26e400d4a0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 shutdown_connections 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26ec162b00 0x7f26ec15fae0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26ec147fc0 0x7f26ec141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f26c806c980 0x7f26c806ee40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 --2- 172.21.15.150:0/1920663786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26ec0aef00 0x7f26ec15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.764+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 >> 172.21.15.150:0/1920663786 conn(0x7f26ec0b23a0 msgr2=0x7f26ec0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.765+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 shutdown_connections 2024-04-15T15:25:48.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.765+0000 7f26eb200640 1 -- 172.21.15.150:0/1920663786 wait complete. 2024-04-15T15:25:48.787 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:25:49.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.999+0000 7fceee000640 1 Processor -- start 2024-04-15T15:25:49.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:48.999+0000 7fceee000640 1 -- start start 2024-04-15T15:25:49.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.000+0000 7fceee000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee80d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.000+0000 7fceee000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcee80d1e00 0x7fcee80b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.000+0000 7fceee000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee80d2c80 con 0x7fcee80d4350 2024-04-15T15:25:49.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.000+0000 7fceee000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fcee80d3850 con 0x7fcee80d1e00 2024-04-15T15:25:49.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.000+0000 7fceee000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcee80d2f20 con 0x7fcee815eb50 2024-04-15T15:25:49.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcee80d1e00 0x7fcee80b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcee80d1e00 0x7fcee80b7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49578/0 (socket says 172.21.15.150:49578) 2024-04-15T15:25:49.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7e00640 1 -- 172.21.15.150:0/2881802453 learned_addr learned my addr 172.21.15.150:0/2881802453 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:49.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7400640 1 --2- 172.21.15.150:0/2881802453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee80d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7e00640 1 -- 172.21.15.150:0/2881802453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 msgr2=0x7fcee80d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:49.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7e00640 1 --2- 172.21.15.150:0/2881802453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee80d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7e00640 1 -- 172.21.15.150:0/2881802453 >> v1:172.21.15.187:6789/0 conn(0x7fcee80d4350 legacy=0x7fcee80d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:49.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7e00640 1 -- 172.21.15.150:0/2881802453 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee80d3200 con 0x7fcee80d1e00 2024-04-15T15:25:49.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.001+0000 7fcee7400640 1 --2- 172.21.15.150:0/2881802453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee80d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:49.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.002+0000 7fcee7e00640 1 --2- 172.21.15.150:0/2881802453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d1e00 0x7fcee80b7b20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcee80b4340 tx=0x7fcee002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddacd7b1865f87a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.003+0000 7fcee6a00640 1 -- 172.21.15.150:0/2881802453 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0034070 con 0x7fcee80d1e00 2024-04-15T15:25:49.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.003+0000 7fcee6a00640 1 -- 172.21.15.150:0/2881802453 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee0030610 con 0x7fcee80d1e00 2024-04-15T15:25:49.005 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.003+0000 7fcee6a00640 1 -- 172.21.15.150:0/2881802453 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee00309e0 con 0x7fcee80d1e00 2024-04-15T15:25:49.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.004+0000 7fceee000640 1 -- 172.21.15.150:0/2881802453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d1e00 msgr2=0x7fcee80b7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.004+0000 7fceee000640 1 --2- 172.21.15.150:0/2881802453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d1e00 0x7fcee80b7b20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcee80b4340 tx=0x7fcee002b960 comp rx=0 tx=0).stop 2024-04-15T15:25:49.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.004+0000 7fceee000640 1 -- 172.21.15.150:0/2881802453 shutdown_connections 2024-04-15T15:25:49.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.004+0000 7fceee000640 1 --2- 172.21.15.150:0/2881802453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d1e00 0x7fcee80b7b20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.004+0000 7fceee000640 1 --2- 172.21.15.150:0/2881802453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee80d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.004+0000 7fceee000640 1 -- 172.21.15.150:0/2881802453 >> 172.21.15.150:0/2881802453 conn(0x7fcee806d1c0 msgr2=0x7fcee806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:49.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.004+0000 7fceee000640 1 -- 172.21.15.150:0/2881802453 shutdown_connections 2024-04-15T15:25:49.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.004+0000 7fceee000640 1 -- 172.21.15.150:0/2881802453 wait complete. 2024-04-15T15:25:49.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.006+0000 7fceee000640 1 Processor -- start 2024-04-15T15:25:49.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.006+0000 7fceee000640 1 -- start start 2024-04-15T15:25:49.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fceee000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee80d1e00 0x7fcee8153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fceee000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d4350 0x7fcee8161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fceee000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee81617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fceee000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcee80cfed0 con 0x7fcee80d4350 2024-04-15T15:25:49.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fceee000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcee815ef30 con 0x7fcee80d1e00 2024-04-15T15:25:49.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fceee000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcee80b8060 con 0x7fcee815eb50 2024-04-15T15:25:49.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fcedea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d4350 0x7fcee8161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fcedea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d4350 0x7fcee8161280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49586/0 (socket says 172.21.15.150:49586) 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fcedea00640 1 -- 172.21.15.150:0/3104077836 learned_addr learned my addr 172.21.15.150:0/3104077836 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fcee7e00640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee81617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.008+0000 7fcee7400640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee80d1e00 0x7fcee8153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedea00640 1 -- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 msgr2=0x7fcee81617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedea00640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee81617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedea00640 1 -- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee80d1e00 msgr2=0x7fcee8153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedea00640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee80d1e00 0x7fcee8153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedea00640 1 -- 172.21.15.150:0/3104077836 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee80ac0b0 con 0x7fcee80d4350 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcee7e00640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee81617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcee7400640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee80d1e00 0x7fcee8153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedea00640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d4350 0x7fcee8161280 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fced000c430 tx=0x7fced000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedfe00640 1 -- 172.21.15.150:0/3104077836 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0011790 con 0x7fcee80d4350 2024-04-15T15:25:49.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedfe00640 1 -- 172.21.15.150:0/3104077836 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced0017070 con 0x7fcee80d4350 2024-04-15T15:25:49.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fcedfe00640 1 -- 172.21.15.150:0/3104077836 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0011bb0 con 0x7fcee80d4350 2024-04-15T15:25:49.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee80cd450 con 0x7fcee80d4350 2024-04-15T15:25:49.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.009+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee80cda10 con 0x7fcee80d4350 2024-04-15T15:25:49.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.010+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee8068c80 con 0x7fcee80d4350 2024-04-15T15:25:49.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.011+0000 7fcedfe00640 1 -- 172.21.15.150:0/3104077836 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fced001c460 con 0x7fcee80d4350 2024-04-15T15:25:49.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.012+0000 7fcedfe00640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcec406c980 0x7fcec406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.012+0000 7fcedfe00640 1 -- 172.21.15.150:0/3104077836 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fced008ba70 con 0x7fcee80d4350 2024-04-15T15:25:49.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.013+0000 7fcee7400640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcec406c980 0x7fcec406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.014+0000 7fcee7400640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcec406c980 0x7fcec406ee40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fced8007f40 tx=0x7fced8007490 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.017+0000 7fcedfe00640 1 -- 172.21.15.150:0/3104077836 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fced0059ee0 con 0x7fcee80d4350 2024-04-15T15:25:49.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.217+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcee80b3aa0 con 0x7fcee80d4350 2024-04-15T15:25:49.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.218+0000 7fcedfe00640 1 -- 172.21.15.150:0/3104077836 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7fced00597e0 con 0x7fcee80d4350 2024-04-15T15:25:49.220 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:25:49.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcec406c980 msgr2=0x7fcec406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcec406c980 0x7fcec406ee40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fced8007f40 tx=0x7fced8007490 comp rx=0 tx=0).stop 2024-04-15T15:25:49.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d4350 msgr2=0x7fcee8161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d4350 0x7fcee8161280 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fced000c430 tx=0x7fced000c8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 shutdown_connections 2024-04-15T15:25:49.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcec406c980 0x7fcec406ee40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee815eb50 0x7fcee81617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcee80d4350 0x7fcee8161280 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 --2- 172.21.15.150:0/3104077836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee80d1e00 0x7fcee8153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 >> 172.21.15.150:0/3104077836 conn(0x7fcee806d1c0 msgr2=0x7fcee80ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:49.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.221+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 shutdown_connections 2024-04-15T15:25:49.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.222+0000 7fceee000640 1 -- 172.21.15.150:0/3104077836 wait complete. 2024-04-15T15:25:49.243 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:25:49.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.452+0000 7fce64000640 1 Processor -- start 2024-04-15T15:25:49.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.453+0000 7fce64000640 1 -- start start 2024-04-15T15:25:49.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.454+0000 7fce64000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fce5c15a440 0x7fce5c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.454+0000 7fce64000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce5c06fc80 0x7fce5c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.454+0000 7fce64000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce5c15d520 con 0x7fce5c0cd620 2024-04-15T15:25:49.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.454+0000 7fce64000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fce5c06e280 con 0x7fce5c06fc80 2024-04-15T15:25:49.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.454+0000 7fce64000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fce5c15d800 con 0x7fce5c15a440 2024-04-15T15:25:49.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.455+0000 7fce60c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce5c06fc80 0x7fce5c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.455+0000 7fce60c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce5c06fc80 0x7fce5c0cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49590/0 (socket says 172.21.15.150:49590) 2024-04-15T15:25:49.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.455+0000 7fce60c00640 1 -- 172.21.15.150:0/532741864 learned_addr learned my addr 172.21.15.150:0/532741864 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:49.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.455+0000 7fce61600640 1 --2- 172.21.15.150:0/532741864 >> v2:172.21.15.187:3300/0 conn(0x7fce5c15a440 0x7fce5c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.455+0000 7fce60c00640 1 -- 172.21.15.150:0/532741864 >> v2:172.21.15.187:3300/0 conn(0x7fce5c15a440 msgr2=0x7fce5c15cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:49.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.455+0000 7fce60c00640 1 --2- 172.21.15.150:0/532741864 >> v2:172.21.15.187:3300/0 conn(0x7fce5c15a440 0x7fce5c15cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.455+0000 7fce60c00640 1 -- 172.21.15.150:0/532741864 >> v1:172.21.15.187:6789/0 conn(0x7fce5c0cd620 legacy=0x7fce5c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:49.457 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.455+0000 7fce60c00640 1 -- 172.21.15.150:0/532741864 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c15dae0 con 0x7fce5c06fc80 2024-04-15T15:25:49.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.456+0000 7fce60c00640 1 --2- 172.21.15.150:0/532741864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c06fc80 0x7fce5c0cd0e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce5c06c9a0 tx=0x7fce4802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3a30b64dc09ef server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce5be00640 1 -- 172.21.15.150:0/532741864 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4802e070 con 0x7fce5c06fc80 2024-04-15T15:25:49.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce5be00640 1 -- 172.21.15.150:0/532741864 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce48037070 con 0x7fce5c06fc80 2024-04-15T15:25:49.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce5be00640 1 -- 172.21.15.150:0/532741864 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48031a00 con 0x7fce5c06fc80 2024-04-15T15:25:49.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce64000640 1 -- 172.21.15.150:0/532741864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c06fc80 msgr2=0x7fce5c0cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce64000640 1 --2- 172.21.15.150:0/532741864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c06fc80 0x7fce5c0cd0e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce5c06c9a0 tx=0x7fce4802b880 comp rx=0 tx=0).stop 2024-04-15T15:25:49.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce64000640 1 -- 172.21.15.150:0/532741864 shutdown_connections 2024-04-15T15:25:49.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce64000640 1 --2- 172.21.15.150:0/532741864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c06fc80 0x7fce5c0cd0e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce64000640 1 --2- 172.21.15.150:0/532741864 >> v2:172.21.15.187:3300/0 conn(0x7fce5c15a440 0x7fce5c15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.457+0000 7fce64000640 1 -- 172.21.15.150:0/532741864 >> 172.21.15.150:0/532741864 conn(0x7fce5c0b2460 msgr2=0x7fce5c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:49.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.458+0000 7fce64000640 1 -- 172.21.15.150:0/532741864 shutdown_connections 2024-04-15T15:25:49.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.458+0000 7fce64000640 1 -- 172.21.15.150:0/532741864 wait complete. 2024-04-15T15:25:49.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.459+0000 7fce64000640 1 Processor -- start 2024-04-15T15:25:49.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.459+0000 7fce64000640 1 -- start start 2024-04-15T15:25:49.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.460+0000 7fce64000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce5c06fc80 0x7fce5c143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.460+0000 7fce64000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c0cd620 0x7fce5c1e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.460+0000 7fce64000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5c15a440 0x7fce5c13ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.460+0000 7fce64000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c06f510 con 0x7fce5c0cd620 2024-04-15T15:25:49.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.460+0000 7fce64000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c15ddc0 con 0x7fce5c06fc80 2024-04-15T15:25:49.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.460+0000 7fce64000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c072860 con 0x7fce5c15a440 2024-04-15T15:25:49.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.461+0000 7fce60c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c0cd620 0x7fce5c1e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.461+0000 7fce60c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c0cd620 0x7fce5c1e6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49598/0 (socket says 172.21.15.150:49598) 2024-04-15T15:25:49.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.461+0000 7fce60c00640 1 -- 172.21.15.150:0/2086566810 learned_addr learned my addr 172.21.15.150:0/2086566810 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.461+0000 7fce60c00640 1 -- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5c15a440 msgr2=0x7fce5c13ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.461+0000 7fce60c00640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5c15a440 0x7fce5c13ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.461+0000 7fce60c00640 1 -- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce5c06fc80 msgr2=0x7fce5c143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.461+0000 7fce60c00640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce5c06fc80 0x7fce5c143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.461+0000 7fce60c00640 1 -- 172.21.15.150:0/2086566810 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c142740 con 0x7fce5c0cd620 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.462+0000 7fce60c00640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c0cd620 0x7fce5c1e6f00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce48000c00 tx=0x7fce480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.462+0000 7fce59600640 1 -- 172.21.15.150:0/2086566810 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48032880 con 0x7fce5c0cd620 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.462+0000 7fce59600640 1 -- 172.21.15.150:0/2086566810 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce48032a20 con 0x7fce5c0cd620 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.462+0000 7fce59600640 1 -- 172.21.15.150:0/2086566810 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48032e10 con 0x7fce5c0cd620 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.462+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c142a20 con 0x7fce5c0cd620 2024-04-15T15:25:49.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.462+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c142fe0 con 0x7fce5c0cd620 2024-04-15T15:25:49.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.463+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5c068c80 con 0x7fce5c0cd620 2024-04-15T15:25:49.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.464+0000 7fce59600640 1 -- 172.21.15.150:0/2086566810 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fce4804c060 con 0x7fce5c0cd620 2024-04-15T15:25:49.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.465+0000 7fce59600640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fce3c06c980 0x7fce3c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.465+0000 7fce59600640 1 -- 172.21.15.150:0/2086566810 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fce480adb20 con 0x7fce5c0cd620 2024-04-15T15:25:49.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.466+0000 7fce61600640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fce3c06c980 0x7fce3c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.467+0000 7fce61600640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fce3c06c980 0x7fce3c06ee40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce50006fd0 tx=0x7fce50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.470+0000 7fce59600640 1 -- 172.21.15.150:0/2086566810 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce48040070 con 0x7fce5c0cd620 2024-04-15T15:25:49.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.670+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce5c15fd20 con 0x7fce5c0cd620 2024-04-15T15:25:49.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.671+0000 7fce59600640 1 -- 172.21.15.150:0/2086566810 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7fce48039310 con 0x7fce5c0cd620 2024-04-15T15:25:49.672 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:25:49.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fce3c06c980 msgr2=0x7fce3c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fce3c06c980 0x7fce3c06ee40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce50006fd0 tx=0x7fce50008040 comp rx=0 tx=0).stop 2024-04-15T15:25:49.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c0cd620 msgr2=0x7fce5c1e6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c0cd620 0x7fce5c1e6f00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce48000c00 tx=0x7fce480046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 shutdown_connections 2024-04-15T15:25:49.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fce3c06c980 0x7fce3c06ee40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce5c15a440 0x7fce5c13ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5c0cd620 0x7fce5c1e6f00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 --2- 172.21.15.150:0/2086566810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce5c06fc80 0x7fce5c143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.673+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 >> 172.21.15.150:0/2086566810 conn(0x7fce5c0b2460 msgr2=0x7fce5c0cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:49.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.674+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 shutdown_connections 2024-04-15T15:25:49.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.674+0000 7fce64000640 1 -- 172.21.15.150:0/2086566810 wait complete. 2024-04-15T15:25:49.695 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-15T15:25:49.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.912+0000 7fe2dba00640 1 Processor -- start 2024-04-15T15:25:49.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.912+0000 7fe2dba00640 1 -- start start 2024-04-15T15:25:49.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.914+0000 7fe2dba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d4072100 0x7fe2d4072580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.914+0000 7fe2dba00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2d406f3f0 0x7fe2d406f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.914+0000 7fe2dba00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d415d200 con 0x7fe2d413f6a0 2024-04-15T15:25:49.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.914+0000 7fe2dba00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d40a95b0 con 0x7fe2d406f3f0 2024-04-15T15:25:49.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.914+0000 7fe2dba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d415d4e0 con 0x7fe2d4072100 2024-04-15T15:25:49.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.915+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d4072100 0x7fe2d4072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.915+0000 7fe2d9a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2d406f3f0 0x7fe2d406f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.915+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d4072100 0x7fe2d4072580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:39550/0 (socket says 172.21.15.150:39550) 2024-04-15T15:25:49.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.915+0000 7fe2d3e00640 1 -- 172.21.15.150:0/1981942874 learned_addr learned my addr 172.21.15.150:0/1981942874 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:49.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.915+0000 7fe2d3e00640 1 -- 172.21.15.150:0/1981942874 >> v2:172.21.15.187:3300/0 conn(0x7fe2d406f3f0 msgr2=0x7fe2d406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:49.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.915+0000 7fe2d3e00640 1 --2- 172.21.15.150:0/1981942874 >> v2:172.21.15.187:3300/0 conn(0x7fe2d406f3f0 0x7fe2d406f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.915+0000 7fe2d3e00640 1 -- 172.21.15.150:0/1981942874 >> v1:172.21.15.187:6789/0 conn(0x7fe2d413f6a0 legacy=0x7fe2d40ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.915+0000 7fe2d3e00640 1 -- 172.21.15.150:0/1981942874 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d415d7c0 con 0x7fe2d4072100 2024-04-15T15:25:49.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.917+0000 7fe2d3e00640 1 --2- 172.21.15.150:0/1981942874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d4072100 0x7fe2d4072580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009870 tx=0x7fe2c002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cccb6f194cd2456 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.917+0000 7fe2d3400640 1 -- 172.21.15.150:0/1981942874 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0034070 con 0x7fe2d4072100 2024-04-15T15:25:49.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.918+0000 7fe2d3400640 1 -- 172.21.15.150:0/1981942874 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c0030840 con 0x7fe2d4072100 2024-04-15T15:25:49.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.918+0000 7fe2d3400640 1 -- 172.21.15.150:0/1981942874 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0030c10 con 0x7fe2d4072100 2024-04-15T15:25:49.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.918+0000 7fe2dba00640 1 -- 172.21.15.150:0/1981942874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d4072100 msgr2=0x7fe2d4072580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.918+0000 7fe2dba00640 1 --2- 172.21.15.150:0/1981942874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d4072100 0x7fe2d4072580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009870 tx=0x7fe2c002b980 comp rx=0 tx=0).stop 2024-04-15T15:25:49.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.918+0000 7fe2dba00640 1 -- 172.21.15.150:0/1981942874 shutdown_connections 2024-04-15T15:25:49.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.918+0000 7fe2dba00640 1 --2- 172.21.15.150:0/1981942874 >> v2:172.21.15.187:3300/0 conn(0x7fe2d406f3f0 0x7fe2d406f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.919+0000 7fe2dba00640 1 --2- 172.21.15.150:0/1981942874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d4072100 0x7fe2d4072580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.919+0000 7fe2dba00640 1 -- 172.21.15.150:0/1981942874 >> 172.21.15.150:0/1981942874 conn(0x7fe2d40b2460 msgr2=0x7fe2d40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:49.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.919+0000 7fe2dba00640 1 -- 172.21.15.150:0/1981942874 shutdown_connections 2024-04-15T15:25:49.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.919+0000 7fe2dba00640 1 -- 172.21.15.150:0/1981942874 wait complete. 2024-04-15T15:25:49.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.920+0000 7fe2dba00640 1 Processor -- start 2024-04-15T15:25:49.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.921+0000 7fe2dba00640 1 -- start start 2024-04-15T15:25:49.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.921+0000 7fe2dba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d406f3f0 0x7fe2d4141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2dba00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2d4072100 0x7fe2d41566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2dba00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2d413f6a0 0x7fe2d4133b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2dba00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d415dd80 con 0x7fe2d4072100 2024-04-15T15:25:49.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2dba00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d413fef0 con 0x7fe2d413f6a0 2024-04-15T15:25:49.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2dba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d415ce60 con 0x7fe2d406f3f0 2024-04-15T15:25:49.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2d4072100 0x7fe2d41566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2d4072100 0x7fe2d41566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49600/0 (socket says 172.21.15.150:49600) 2024-04-15T15:25:49.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d3e00640 1 -- 172.21.15.150:0/446120423 learned_addr learned my addr 172.21.15.150:0/446120423 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:49.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d9000640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d406f3f0 0x7fe2d4141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d9a00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2d413f6a0 0x7fe2d4133b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d3e00640 1 -- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d406f3f0 msgr2=0x7fe2d4141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d3e00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d406f3f0 0x7fe2d4141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d3e00640 1 -- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2d413f6a0 msgr2=0x7fe2d4133b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d3e00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2d413f6a0 0x7fe2d4133b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.922+0000 7fe2d3e00640 1 -- 172.21.15.150:0/446120423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d4140680 con 0x7fe2d4072100 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.923+0000 7fe2d9000640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d406f3f0 0x7fe2d4141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.923+0000 7fe2d3e00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2d4072100 0x7fe2d41566b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe2c002b9c0 tx=0x7fe2c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.923+0000 7fe2d0c00640 1 -- 172.21.15.150:0/446120423 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0048070 con 0x7fe2d4072100 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.923+0000 7fe2d0c00640 1 -- 172.21.15.150:0/446120423 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c0030640 con 0x7fe2d4072100 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.923+0000 7fe2d0c00640 1 -- 172.21.15.150:0/446120423 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0034040 con 0x7fe2d4072100 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.923+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d4140960 con 0x7fe2d4072100 2024-04-15T15:25:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.923+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d4140f20 con 0x7fe2d4072100 2024-04-15T15:25:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.924+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d4068c80 con 0x7fe2d4072100 2024-04-15T15:25:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.925+0000 7fe2d0c00640 1 -- 172.21.15.150:0/446120423 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe2c0037070 con 0x7fe2d4072100 2024-04-15T15:25:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.927+0000 7fe2d0c00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe2b406c980 0x7fe2b406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.927+0000 7fe2d0c00640 1 -- 172.21.15.150:0/446120423 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fe2c00ad8c0 con 0x7fe2d4072100 2024-04-15T15:25:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.927+0000 7fe2d9000640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe2b406c980 0x7fe2b406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.929+0000 7fe2d9000640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe2b406c980 0x7fe2b406ee40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8001680 tx=0x7fe2c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:49.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:49.932+0000 7fe2d0c00640 1 -- 172.21.15.150:0/446120423 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2c007bc20 con 0x7fe2d4072100 2024-04-15T15:25:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.146+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe2d4136210 con 0x7fe2d4072100 2024-04-15T15:25:50.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.147+0000 7fe2d0c00640 1 -- 172.21.15.150:0/446120423 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+124 (secure 0 0 0) 0x7fe2c007b520 con 0x7fe2d4072100 2024-04-15T15:25:50.148 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6835/3327940880 2024-04-16T15:25:33.990360+0000 2024-04-15T15:25:50.148 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6833/3327940880 2024-04-16T15:25:33.990360+0000 2024-04-15T15:25:50.149 INFO:teuthology.orchestra.run.smithi150.stderr:listed 2 entries 2024-04-15T15:25:50.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe2b406c980 msgr2=0x7fe2b406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe2b406c980 0x7fe2b406ee40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8001680 tx=0x7fe2c8009040 comp rx=0 tx=0).stop 2024-04-15T15:25:50.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2d4072100 msgr2=0x7fe2d41566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2d4072100 0x7fe2d41566b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe2c002b9c0 tx=0x7fe2c0004730 comp rx=0 tx=0).stop 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 shutdown_connections 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2d413f6a0 0x7fe2d4133b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2d4072100 0x7fe2d41566b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fe2b406c980 0x7fe2b406ee40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 --2- 172.21.15.150:0/446120423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2d406f3f0 0x7fe2d4141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.149+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 >> 172.21.15.150:0/446120423 conn(0x7fe2d40b2460 msgr2=0x7fe2d40b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.150+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 shutdown_connections 2024-04-15T15:25:50.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.150+0000 7fe2dba00640 1 -- 172.21.15.150:0/446120423 wait complete. 2024-04-15T15:25:50.171 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-15T15:25:50.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:50.172 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.3 2024-04-15T15:25:50.285 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:25:50.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:25:50.285 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-15T15:25:50.336 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:50.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.557+0000 7f8ee3e00640 1 Processor -- start 2024-04-15T15:25:50.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.557+0000 7f8ee3e00640 1 -- start start 2024-04-15T15:25:50.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.559+0000 7f8ee3e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee40cee00 0x7f8ee406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:50.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.559+0000 7f8ee3e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee415dbb0 con 0x7f8ee4071f50 2024-04-15T15:25:50.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.559+0000 7f8ee3e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee415d8d0 con 0x7f8ee415d2f0 2024-04-15T15:25:50.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.559+0000 7f8ee3e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee406f4f0 con 0x7f8ee40cee00 2024-04-15T15:25:50.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.559+0000 7f8ee2000640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f8ee4071f50 0x7f8ee415cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:53696/0 (socket says 172.21.15.150:53696) 2024-04-15T15:25:50.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.559+0000 7f8ee2000640 1 -- 172.21.15.150:0/1151990484 learned_addr learned my addr 172.21.15.150:0/1151990484 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4292637513 0 0) 0x7f8ee415dbb0 con 0x7f8ee4071f50 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee2a00640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee40cee00 0x7f8ee406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ec8003520 con 0x7f8ee4071f50 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4157010620 0 0) 0x7f8ec8003520 con 0x7f8ee4071f50 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee40cee00 msgr2=0x7f8ee406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee40cee00 0x7f8ee406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 >> v1:172.21.15.187:6789/0 conn(0x7f8ee415d2f0 legacy=0x7f8ee415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee406dde0 con 0x7f8ee4071f50 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f8ed0001670 con 0x7f8ee4071f50 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 >> v1:172.21.15.150:6789/0 conn(0x7f8ee4071f50 legacy=0x7f8ee415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ec80039e0 0x7f8ec8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:50.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee40cee00 0x7f8ec8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:50.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:50.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec800bb50 con 0x7f8ec8004d40 2024-04-15T15:25:50.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec800bcd0 con 0x7f8ec80039e0 2024-04-15T15:25:50.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec800bea0 con 0x7f8ee40cee00 2024-04-15T15:25:50.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.560+0000 7f8ee3400640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:50.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.561+0000 7f8ee2000640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee40cee00 0x7f8ec8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:50.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.561+0000 7f8ee2a00640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ec80039e0 0x7f8ec8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:50.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.561+0000 7f8ee3400640 1 -- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee40cee00 msgr2=0x7f8ec8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:50.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.561+0000 7f8ee3400640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee40cee00 0x7f8ec8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.561+0000 7f8ee3400640 1 -- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ec80039e0 msgr2=0x7f8ec8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:50.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.561+0000 7f8ee3400640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ec80039e0 0x7f8ec8003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.561+0000 7f8ee3400640 1 -- 172.21.15.150:0/1151990484 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec8003520 con 0x7f8ec8004d40 2024-04-15T15:25:50.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.562+0000 7f8ee3400640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001e80 tx=0x7f8ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac885173a16ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:50.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.562+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc001a70 con 0x7f8ec8004d40 2024-04-15T15:25:50.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.562+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8edc003b70 con 0x7f8ec8004d40 2024-04-15T15:25:50.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.562+0000 7f8ee1600640 1 -- 172.21.15.150:0/1151990484 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0050d0 con 0x7f8ec8004d40 2024-04-15T15:25:50.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.563+0000 7f8ee3e00640 1 -- 172.21.15.150:0/1151990484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 msgr2=0x7f8ec80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.563+0000 7f8ee3e00640 1 --2- 172.21.15.150:0/1151990484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001e80 tx=0x7f8ec8005830 comp rx=0 tx=0).stop 2024-04-15T15:25:50.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.563+0000 7f8ee3400640 1 -- 172.21.15.150:0/1151990484 reap_dead start 2024-04-15T15:25:50.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.563+0000 7f8ee3e00640 1 -- 172.21.15.150:0/1151990484 shutdown_connections 2024-04-15T15:25:50.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.563+0000 7f8ee3e00640 1 -- 172.21.15.150:0/1151990484 >> 172.21.15.150:0/1151990484 conn(0x7f8ee406b930 msgr2=0x7f8ee406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:50.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.563+0000 7f8ee3e00640 1 -- 172.21.15.150:0/1151990484 shutdown_connections 2024-04-15T15:25:50.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.563+0000 7f8ee3e00640 1 -- 172.21.15.150:0/1151990484 wait complete. 2024-04-15T15:25:50.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.565+0000 7f8ee3e00640 1 Processor -- start 2024-04-15T15:25:50.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.565+0000 7f8ee3e00640 1 -- start start 2024-04-15T15:25:50.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ee41537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:50.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee3e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee4140430 0x7f8ee413d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:50.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee3e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee413d960 0x7f8ee41563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:50.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee406e8c0 con 0x7f8ec8004d40 2024-04-15T15:25:50.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee3e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee406e3a0 con 0x7f8ee4140430 2024-04-15T15:25:50.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee3e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee406ea40 con 0x7f8ee413d960 2024-04-15T15:25:50.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee2a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ee41537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:50.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee2a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ee41537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49622/0 (socket says 172.21.15.150:49622) 2024-04-15T15:25:50.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.566+0000 7f8ee2a00640 1 -- 172.21.15.150:0/11946639 learned_addr learned my addr 172.21.15.150:0/11946639 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:50.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8ee2a00640 1 -- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee413d960 msgr2=0x7f8ee41563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8ee2a00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee413d960 0x7f8ee41563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8ee2a00640 1 -- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee4140430 msgr2=0x7f8ee413d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8ee2a00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee4140430 0x7f8ee413d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8ee2a00640 1 -- 172.21.15.150:0/11946639 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee413d670 con 0x7f8ec8004d40 2024-04-15T15:25:50.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8ee2a00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ee41537a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400e7e0 tx=0x7f8ed400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:50.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8edaa00640 1 -- 172.21.15.150:0/11946639 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4017070 con 0x7f8ec8004d40 2024-04-15T15:25:50.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8edaa00640 1 -- 172.21.15.150:0/11946639 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed400c650 con 0x7f8ec8004d40 2024-04-15T15:25:50.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee415f7f0 con 0x7f8ec8004d40 2024-04-15T15:25:50.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8edaa00640 1 -- 172.21.15.150:0/11946639 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400f040 con 0x7f8ec8004d40 2024-04-15T15:25:50.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.567+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee415fdb0 con 0x7f8ec8004d40 2024-04-15T15:25:50.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.568+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee40755c0 con 0x7f8ec8004d40 2024-04-15T15:25:50.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.569+0000 7f8edaa00640 1 -- 172.21.15.150:0/11946639 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8ed401a070 con 0x7f8ec8004d40 2024-04-15T15:25:50.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.570+0000 7f8edaa00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ec406c9b0 0x7f8ec406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:50.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.570+0000 7f8edaa00640 1 -- 172.21.15.150:0/11946639 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f8ed408b980 con 0x7f8ec8004d40 2024-04-15T15:25:50.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.571+0000 7f8ee2000640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ec406c9b0 0x7f8ec406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:50.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.572+0000 7f8ee2000640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ec406c9b0 0x7f8ec406ee70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40b1670 tx=0x7f8ed0001d90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:50.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.575+0000 7f8edaa00640 1 -- 172.21.15.150:0/11946639 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ed4059be0 con 0x7f8ec8004d40 2024-04-15T15:25:50.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.812+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ee415b460 con 0x7f8ec8004d40 2024-04-15T15:25:50.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.813+0000 7f8edaa00640 1 -- 172.21.15.150:0/11946639 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5194 (secure 0 0 0) 0x7f8ed40594e0 con 0x7f8ec8004d40 2024-04-15T15:25:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:50.815 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:37.899562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5499":{"gid":5499,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6835/3243584315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3243584315},{"type":"v1","addr":"172.21.15.150:6835","nonce":3243584315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5499,"qdb_cluster":[5499]},"id":2}]} 2024-04-15T15:25:50.816 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 22 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ec406c9b0 msgr2=0x7f8ec406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ec406c9b0 0x7f8ec406ee70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40b1670 tx=0x7f8ed0001d90 comp rx=0 tx=0).stop 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 msgr2=0x7f8ee41537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ee41537a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400e7e0 tx=0x7f8ed400eca0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 shutdown_connections 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ec406c9b0 0x7f8ec406ee70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ee413d960 0x7f8ee41563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee4140430 0x7f8ee413d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 --2- 172.21.15.150:0/11946639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ee41537a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:50.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 >> 172.21.15.150:0/11946639 conn(0x7f8ee406b930 msgr2=0x7f8ee40cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:50.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 shutdown_connections 2024-04-15T15:25:50.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:50.816+0000 7f8ee3e00640 1 -- 172.21.15.150:0/11946639 wait complete. 2024-04-15T15:25:50.836 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:51.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.040+0000 7efc51000640 1 Processor -- start 2024-04-15T15:25:51.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.040+0000 7efc51000640 1 -- start start 2024-04-15T15:25:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.042+0000 7efc51000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efc4c1481f0 0x7efc4c15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.042+0000 7efc51000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc4c0728f0 0x7efc4c1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.042+0000 7efc51000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc4c1485d0 con 0x7efc4c0affc0 2024-04-15T15:25:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.042+0000 7efc51000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7efc4c15ffc0 con 0x7efc4c1481f0 2024-04-15T15:25:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.042+0000 7efc51000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efc4c0aef20 con 0x7efc4c0728f0 2024-04-15T15:25:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.042+0000 7efc49e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efc4c1481f0 0x7efc4c15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.042+0000 7efc49e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efc4c1481f0 0x7efc4c15fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49628/0 (socket says 172.21.15.150:49628) 2024-04-15T15:25:51.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.042+0000 7efc49e00640 1 -- 172.21.15.150:0/2269967949 learned_addr learned my addr 172.21.15.150:0/2269967949 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:51.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.043+0000 7efc49e00640 1 -- 172.21.15.150:0/2269967949 >> v2:172.21.15.187:3300/0 conn(0x7efc4c0728f0 msgr2=0x7efc4c1601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:51.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.043+0000 7efc4a800640 1 --2- 172.21.15.150:0/2269967949 >> v2:172.21.15.187:3300/0 conn(0x7efc4c0728f0 0x7efc4c1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.043+0000 7efc49e00640 1 --2- 172.21.15.150:0/2269967949 >> v2:172.21.15.187:3300/0 conn(0x7efc4c0728f0 0x7efc4c1601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.043+0000 7efc49e00640 1 -- 172.21.15.150:0/2269967949 >> v1:172.21.15.187:6789/0 conn(0x7efc4c0affc0 legacy=0x7efc4c0721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:51.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.043+0000 7efc49e00640 1 -- 172.21.15.150:0/2269967949 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c0af200 con 0x7efc4c1481f0 2024-04-15T15:25:51.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.044+0000 7efc49e00640 1 --2- 172.21.15.150:0/2269967949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c1481f0 0x7efc4c15fa80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0ada90 tx=0x7efc3c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3789134f32acd31 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.044+0000 7efc49400640 1 -- 172.21.15.150:0/2269967949 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c02e070 con 0x7efc4c1481f0 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.044+0000 7efc49400640 1 -- 172.21.15.150:0/2269967949 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc3c037070 con 0x7efc4c1481f0 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.044+0000 7efc49400640 1 -- 172.21.15.150:0/2269967949 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c031a00 con 0x7efc4c1481f0 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.045+0000 7efc51000640 1 -- 172.21.15.150:0/2269967949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c1481f0 msgr2=0x7efc4c15fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.045+0000 7efc51000640 1 --2- 172.21.15.150:0/2269967949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c1481f0 0x7efc4c15fa80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0ada90 tx=0x7efc3c02b880 comp rx=0 tx=0).stop 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.045+0000 7efc51000640 1 -- 172.21.15.150:0/2269967949 shutdown_connections 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.045+0000 7efc51000640 1 --2- 172.21.15.150:0/2269967949 >> v2:172.21.15.187:3300/0 conn(0x7efc4c0728f0 0x7efc4c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.045+0000 7efc51000640 1 --2- 172.21.15.150:0/2269967949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c1481f0 0x7efc4c15fa80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.045+0000 7efc51000640 1 -- 172.21.15.150:0/2269967949 >> 172.21.15.150:0/2269967949 conn(0x7efc4c0b27b0 msgr2=0x7efc4c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.045+0000 7efc51000640 1 -- 172.21.15.150:0/2269967949 shutdown_connections 2024-04-15T15:25:51.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.045+0000 7efc51000640 1 -- 172.21.15.150:0/2269967949 wait complete. 2024-04-15T15:25:51.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.047+0000 7efc51000640 1 Processor -- start 2024-04-15T15:25:51.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.047+0000 7efc51000640 1 -- start start 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc51000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c0728f0 0x7efc4c0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc51000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc4c0affc0 0x7efc4c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc51000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc4c1481f0 0x7efc4c1f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc51000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c0afaa0 con 0x7efc4c0728f0 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc51000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c0af7c0 con 0x7efc4c1481f0 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc51000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc4c0b5560 con 0x7efc4c0affc0 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc49e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c0728f0 0x7efc4c0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc49e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c0728f0 0x7efc4c0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49642/0 (socket says 172.21.15.150:49642) 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc49e00640 1 -- 172.21.15.150:0/354088299 learned_addr learned my addr 172.21.15.150:0/354088299 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc41400640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc4c0affc0 0x7efc4c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc4a800640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc4c1481f0 0x7efc4c1f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc49e00640 1 -- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc4c0affc0 msgr2=0x7efc4c0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc49e00640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc4c0affc0 0x7efc4c0ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc49e00640 1 -- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc4c1481f0 msgr2=0x7efc4c1f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc49e00640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc4c1481f0 0x7efc4c1f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc49e00640 1 -- 172.21.15.150:0/354088299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c0cec90 con 0x7efc4c0728f0 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.048+0000 7efc41400640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc4c0affc0 0x7efc4c0ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.049+0000 7efc49e00640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c0728f0 0x7efc4c0ce210 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000c00 tx=0x7efc3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.049+0000 7efc42a00640 1 -- 172.21.15.150:0/354088299 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c032880 con 0x7efc4c0728f0 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.049+0000 7efc42a00640 1 -- 172.21.15.150:0/354088299 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc3c032a20 con 0x7efc4c0728f0 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.049+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c1f59a0 con 0x7efc4c0728f0 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.049+0000 7efc42a00640 1 -- 172.21.15.150:0/354088299 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c032e10 con 0x7efc4c0728f0 2024-04-15T15:25:51.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.049+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1f5f60 con 0x7efc4c0728f0 2024-04-15T15:25:51.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.050+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c068c80 con 0x7efc4c0728f0 2024-04-15T15:25:51.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.051+0000 7efc42a00640 1 -- 172.21.15.150:0/354088299 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7efc3c04c060 con 0x7efc4c0728f0 2024-04-15T15:25:51.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.053+0000 7efc42a00640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc2806c980 0x7efc2806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.053+0000 7efc42a00640 1 -- 172.21.15.150:0/354088299 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7efc3c0adaf0 con 0x7efc4c0728f0 2024-04-15T15:25:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.053+0000 7efc41400640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc2806c980 0x7efc2806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.055+0000 7efc41400640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc2806c980 0x7efc2806ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efc34007fd0 tx=0x7efc34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.057+0000 7efc42a00640 1 -- 172.21.15.150:0/354088299 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc3c040070 con 0x7efc4c0728f0 2024-04-15T15:25:51.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.260+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc4c06d1e0 con 0x7efc4c0728f0 2024-04-15T15:25:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.261+0000 7efc42a00640 1 -- 172.21.15.150:0/354088299 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16848 (secure 0 0 0) 0x7efc3c07be50 con 0x7efc4c0728f0 2024-04-15T15:25:51.264 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:51.264 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":36,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:33.990388+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/3327940880":"2024-04-16T15:25:33.990360+0000","172.21.15.150:6833/3327940880":"2024-04-16T15:25:33.990360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc2806c980 msgr2=0x7efc2806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc2806c980 0x7efc2806ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efc34007fd0 tx=0x7efc34009040 comp rx=0 tx=0).stop 2024-04-15T15:25:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c0728f0 msgr2=0x7efc4c0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c0728f0 0x7efc4c0ce210 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000c00 tx=0x7efc3c0046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 shutdown_connections 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc4c1481f0 0x7efc4c1f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efc2806c980 0x7efc2806ee40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc4c0affc0 0x7efc4c0ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 --2- 172.21.15.150:0/354088299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc4c0728f0 0x7efc4c0ce210 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.264+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 >> 172.21.15.150:0/354088299 conn(0x7efc4c0b27b0 msgr2=0x7efc4c0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.265+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 shutdown_connections 2024-04-15T15:25:51.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.265+0000 7efc51000640 1 -- 172.21.15.150:0/354088299 wait complete. 2024-04-15T15:25:51.285 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-15T15:23:43.252042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-15T15:25:04.190900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-15T15:25:05.324314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-15T15:25:51.285 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-15T15:25:51.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.487+0000 7fbb0fc00640 1 Processor -- start 2024-04-15T15:25:51.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.487+0000 7fbb0fc00640 1 -- start start 2024-04-15T15:25:51.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.489+0000 7fbb0fc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb0815a440 0x7fbb0815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.489+0000 7fbb0fc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 0x7fbb080cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.489+0000 7fbb0fc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb080cd620 0x7fbb08072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.489+0000 7fbb0fc00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0806e280 con 0x7fbb080cd620 2024-04-15T15:25:51.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.489+0000 7fbb0fc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0815d520 con 0x7fbb0815a440 2024-04-15T15:25:51.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.489+0000 7fbb0fc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0815d800 con 0x7fbb0806fc80 2024-04-15T15:25:51.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.489+0000 7fbb0dc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb080cd620 0x7fbb08072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb0dc00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb080cd620 0x7fbb08072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49648/0 (socket says 172.21.15.150:49648) 2024-04-15T15:25:51.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb0dc00640 1 -- 172.21.15.150:0/95521438 learned_addr learned my addr 172.21.15.150:0/95521438 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:51.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb0d200640 1 --2- 172.21.15.150:0/95521438 >> v2:172.21.15.187:3300/0 conn(0x7fbb0815a440 0x7fbb0815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb07e00640 1 --2- 172.21.15.150:0/95521438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 0x7fbb080cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb0dc00640 1 -- 172.21.15.150:0/95521438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 msgr2=0x7fbb080cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:51.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb0dc00640 1 --2- 172.21.15.150:0/95521438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 0x7fbb080cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb0dc00640 1 -- 172.21.15.150:0/95521438 >> v2:172.21.15.187:3300/0 conn(0x7fbb0815a440 msgr2=0x7fbb0815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:51.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb0dc00640 1 --2- 172.21.15.150:0/95521438 >> v2:172.21.15.187:3300/0 conn(0x7fbb0815a440 0x7fbb0815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.490+0000 7fbb0dc00640 1 -- 172.21.15.150:0/95521438 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0815dae0 con 0x7fbb080cd620 2024-04-15T15:25:51.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.492+0000 7fbb0dc00640 1 --2- 172.21.15.150:0/95521438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb080cd620 0x7fbb08072150 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbb0806c9a0 tx=0x7fbb0002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342a3fab32cbe2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.492+0000 7fbb07400640 1 -- 172.21.15.150:0/95521438 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0002ca70 con 0x7fbb080cd620 2024-04-15T15:25:51.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.492+0000 7fbb07400640 1 -- 172.21.15.150:0/95521438 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0002cc10 con 0x7fbb080cd620 2024-04-15T15:25:51.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.492+0000 7fbb07400640 1 -- 172.21.15.150:0/95521438 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000343f0 con 0x7fbb080cd620 2024-04-15T15:25:51.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 -- 172.21.15.150:0/95521438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb080cd620 msgr2=0x7fbb08072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/95521438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb080cd620 0x7fbb08072150 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbb0806c9a0 tx=0x7fbb0002bed0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 -- 172.21.15.150:0/95521438 shutdown_connections 2024-04-15T15:25:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/95521438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb080cd620 0x7fbb08072150 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/95521438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 0x7fbb080cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/95521438 >> v2:172.21.15.187:3300/0 conn(0x7fbb0815a440 0x7fbb0815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 -- 172.21.15.150:0/95521438 >> 172.21.15.150:0/95521438 conn(0x7fbb080b2460 msgr2=0x7fbb080b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 -- 172.21.15.150:0/95521438 shutdown_connections 2024-04-15T15:25:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.493+0000 7fbb0fc00640 1 -- 172.21.15.150:0/95521438 wait complete. 2024-04-15T15:25:51.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.495+0000 7fbb0fc00640 1 Processor -- start 2024-04-15T15:25:51.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.495+0000 7fbb0fc00640 1 -- start start 2024-04-15T15:25:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0fc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 0x7fbb08133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0fc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb080cd620 0x7fbb081593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0fc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb0815a440 0x7fbb08131dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0fc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0806f510 con 0x7fbb0815a440 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0fc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0815ddc0 con 0x7fbb080cd620 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0fc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb08072690 con 0x7fbb0806fc80 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb0815a440 0x7fbb08131dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb0815a440 0x7fbb08131dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49650/0 (socket says 172.21.15.150:49650) 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0dc00640 1 -- 172.21.15.150:0/3060834751 learned_addr learned my addr 172.21.15.150:0/3060834751 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb0d200640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 0x7fbb08133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.496+0000 7fbb07e00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb080cd620 0x7fbb081593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb0dc00640 1 -- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 msgr2=0x7fbb08133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb0dc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 0x7fbb08133830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb0dc00640 1 -- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb080cd620 msgr2=0x7fbb081593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb0dc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb080cd620 0x7fbb081593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb0dc00640 1 -- 172.21.15.150:0/3060834751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb081598f0 con 0x7fbb0815a440 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb07e00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb080cd620 0x7fbb081593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:51.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb0dc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb0815a440 0x7fbb08131dd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbb00036040 tx=0x7fbb00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb04c00640 1 -- 172.21.15.150:0/3060834751 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00043020 con 0x7fbb0815a440 2024-04-15T15:25:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb04c00640 1 -- 172.21.15.150:0/3060834751 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0003e450 con 0x7fbb0815a440 2024-04-15T15:25:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb08132790 con 0x7fbb0815a440 2024-04-15T15:25:51.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb04c00640 1 -- 172.21.15.150:0/3060834751 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00009510 con 0x7fbb0815a440 2024-04-15T15:25:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.497+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb08132d50 con 0x7fbb0815a440 2024-04-15T15:25:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.499+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb08068c80 con 0x7fbb0815a440 2024-04-15T15:25:51.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.499+0000 7fbb04c00640 1 -- 172.21.15.150:0/3060834751 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fbb00046020 con 0x7fbb0815a440 2024-04-15T15:25:51.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.500+0000 7fbb04c00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbae406c980 0x7fbae406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.500+0000 7fbb04c00640 1 -- 172.21.15.150:0/3060834751 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fbb000ae480 con 0x7fbb0815a440 2024-04-15T15:25:51.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.501+0000 7fbb0d200640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbae406c980 0x7fbae406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.502+0000 7fbb0d200640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbae406c980 0x7fbae406ee40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8006fd0 tx=0x7fbaf8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.505+0000 7fbb04c00640 1 -- 172.21.15.150:0/3060834751 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb0004d030 con 0x7fbb0815a440 2024-04-15T15:25:51.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.704+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb0806d130 con 0x7fbb0815a440 2024-04-15T15:25:51.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.705+0000 7fbb04c00640 1 -- 172.21.15.150:0/3060834751 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7fbb0007c7f0 con 0x7fbb0815a440 2024-04-15T15:25:51.706 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbae406c980 msgr2=0x7fbae406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbae406c980 0x7fbae406ee40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8006fd0 tx=0x7fbaf8008040 comp rx=0 tx=0).stop 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb0815a440 msgr2=0x7fbb08131dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb0815a440 0x7fbb08131dd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbb00036040 tx=0x7fbb00004a40 comp rx=0 tx=0).stop 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 shutdown_connections 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb0815a440 0x7fbb08131dd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb080cd620 0x7fbb081593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbae406c980 0x7fbae406ee40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 --2- 172.21.15.150:0/3060834751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb0806fc80 0x7fbb08133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 >> 172.21.15.150:0/3060834751 conn(0x7fbb080b2460 msgr2=0x7fbb080adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:51.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.707+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 shutdown_connections 2024-04-15T15:25:51.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.708+0000 7fbb0fc00640 1 -- 172.21.15.150:0/3060834751 wait complete. 2024-04-15T15:25:51.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-15T15:25:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.936+0000 7efded200640 1 Processor -- start 2024-04-15T15:25:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.936+0000 7efded200640 1 -- start start 2024-04-15T15:25:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.938+0000 7efded200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efde8145170 0x7efde80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.938+0000 7efded200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efde80ce070 0x7efde80b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.938+0000 7efded200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde80b4410 con 0x7efde80b59f0 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.938+0000 7efded200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7efde8145550 con 0x7efde8145170 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.938+0000 7efded200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efde80b46f0 con 0x7efde80ce070 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.938+0000 7efde5e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efde8145170 0x7efde80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.939+0000 7efde5e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efde8145170 0x7efde80cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49664/0 (socket says 172.21.15.150:49664) 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.939+0000 7efde5e00640 1 -- 172.21.15.150:0/3283606323 learned_addr learned my addr 172.21.15.150:0/3283606323 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.939+0000 7efddd400640 1 --2- 172.21.15.150:0/3283606323 >> v2:172.21.15.187:3300/0 conn(0x7efde80ce070 0x7efde80b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.939+0000 7efde5e00640 1 -- 172.21.15.150:0/3283606323 >> v2:172.21.15.187:3300/0 conn(0x7efde80ce070 msgr2=0x7efde80b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.939+0000 7efde5e00640 1 --2- 172.21.15.150:0/3283606323 >> v2:172.21.15.187:3300/0 conn(0x7efde80ce070 0x7efde80b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.939+0000 7efde5e00640 1 -- 172.21.15.150:0/3283606323 >> v1:172.21.15.187:6789/0 conn(0x7efde80b59f0 legacy=0x7efde806f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.939+0000 7efde5e00640 1 -- 172.21.15.150:0/3283606323 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde80b49d0 con 0x7efde8145170 2024-04-15T15:25:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.939+0000 7efddd400640 1 --2- 172.21.15.150:0/3283606323 >> v2:172.21.15.187:3300/0 conn(0x7efde80ce070 0x7efde80b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:51.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.940+0000 7efde5e00640 1 --2- 172.21.15.150:0/3283606323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde80cdb30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efde81456d0 tx=0x7efdd802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ddadef821796e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.940+0000 7efde5400640 1 -- 172.21.15.150:0/3283606323 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8035020 con 0x7efde8145170 2024-04-15T15:25:51.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efde5400640 1 -- 172.21.15.150:0/3283606323 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd802d040 con 0x7efde8145170 2024-04-15T15:25:51.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efde5400640 1 -- 172.21.15.150:0/3283606323 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8031a80 con 0x7efde8145170 2024-04-15T15:25:51.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efded200640 1 -- 172.21.15.150:0/3283606323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 msgr2=0x7efde80cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efded200640 1 --2- 172.21.15.150:0/3283606323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde80cdb30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efde81456d0 tx=0x7efdd802b900 comp rx=0 tx=0).stop 2024-04-15T15:25:51.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efded200640 1 -- 172.21.15.150:0/3283606323 shutdown_connections 2024-04-15T15:25:51.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efded200640 1 --2- 172.21.15.150:0/3283606323 >> v2:172.21.15.187:3300/0 conn(0x7efde80ce070 0x7efde80b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efded200640 1 --2- 172.21.15.150:0/3283606323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde80cdb30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efded200640 1 -- 172.21.15.150:0/3283606323 >> 172.21.15.150:0/3283606323 conn(0x7efde80b2460 msgr2=0x7efde80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efded200640 1 -- 172.21.15.150:0/3283606323 shutdown_connections 2024-04-15T15:25:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.941+0000 7efded200640 1 -- 172.21.15.150:0/3283606323 wait complete. 2024-04-15T15:25:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.943+0000 7efded200640 1 Processor -- start 2024-04-15T15:25:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.943+0000 7efded200640 1 -- start start 2024-04-15T15:25:51.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.944+0000 7efded200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efde80b59f0 0x7efde813b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.944+0000 7efded200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde80ce070 0x7efde8142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.944+0000 7efded200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde8142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.944+0000 7efded200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efde80b4030 con 0x7efde8145170 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.944+0000 7efded200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efde80b4f90 con 0x7efde80ce070 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.944+0000 7efded200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efde80b41b0 con 0x7efde80b59f0 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde8142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde8142e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49674/0 (socket says 172.21.15.150:49674) 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 -- 172.21.15.150:0/1255847330 learned_addr learned my addr 172.21.15.150:0/1255847330 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde5e00640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efde80b59f0 0x7efde813b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efddd400640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde80ce070 0x7efde8142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 -- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efde80b59f0 msgr2=0x7efde813b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efde80b59f0 0x7efde813b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 -- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde80ce070 msgr2=0x7efde8142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde80ce070 0x7efde8142950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:51.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 -- 172.21.15.150:0/1255847330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde8137540 con 0x7efde8145170 2024-04-15T15:25:51.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.945+0000 7efde6800640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde8142e90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efde000c780 tx=0x7efde000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.946+0000 7efddea00640 1 -- 172.21.15.150:0/1255847330 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0017070 con 0x7efde8145170 2024-04-15T15:25:51.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.946+0000 7efddea00640 1 -- 172.21.15.150:0/1255847330 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde0003da0 con 0x7efde8145170 2024-04-15T15:25:51.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.946+0000 7efddea00640 1 -- 172.21.15.150:0/1255847330 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde00052b0 con 0x7efde8145170 2024-04-15T15:25:51.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.946+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde8137820 con 0x7efde8145170 2024-04-15T15:25:51.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.946+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde813a3e0 con 0x7efde8145170 2024-04-15T15:25:51.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.947+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde8139f10 con 0x7efde8145170 2024-04-15T15:25:51.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.948+0000 7efddea00640 1 -- 172.21.15.150:0/1255847330 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7efde0005450 con 0x7efde8145170 2024-04-15T15:25:51.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.949+0000 7efddea00640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efdc406c980 0x7efdc406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:51.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.949+0000 7efddea00640 1 -- 172.21.15.150:0/1255847330 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7efde008b1d0 con 0x7efde8145170 2024-04-15T15:25:51.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.949+0000 7efde5e00640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efdc406c980 0x7efdc406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:51.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.951+0000 7efde5e00640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efdc406c980 0x7efdc406ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efdd8036000 tx=0x7efdd803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:51.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:51.953+0000 7efddea00640 1 -- 172.21.15.150:0/1255847330 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efde00595b0 con 0x7efde8145170 2024-04-15T15:25:52.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.153+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efde8068c80 con 0x7efde8145170 2024-04-15T15:25:52.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.154+0000 7efddea00640 1 -- 172.21.15.150:0/1255847330 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7efde0058eb0 con 0x7efde8145170 2024-04-15T15:25:52.156 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-04-15T15:25:52.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efdc406c980 msgr2=0x7efdc406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efdc406c980 0x7efdc406ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efdd8036000 tx=0x7efdd803c040 comp rx=0 tx=0).stop 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 msgr2=0x7efde8142e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde8142e90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efde000c780 tx=0x7efde000cc40 comp rx=0 tx=0).stop 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 shutdown_connections 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efde8145170 0x7efde8142e90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde80ce070 0x7efde8142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efdc406c980 0x7efdc406ee40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 --2- 172.21.15.150:0/1255847330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efde80b59f0 0x7efde813b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.156+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 >> 172.21.15.150:0/1255847330 conn(0x7efde80b2460 msgr2=0x7efde8140f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.157+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 shutdown_connections 2024-04-15T15:25:52.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.157+0000 7efded200640 1 -- 172.21.15.150:0/1255847330 wait complete. 2024-04-15T15:25:52.177 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-15T15:25:52.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.379+0000 7f1747e00640 1 Processor -- start 2024-04-15T15:25:52.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.379+0000 7f1747e00640 1 -- start start 2024-04-15T15:25:52.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.380+0000 7f1747e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1748147c90 0x7f174806f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.380+0000 7f1747e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f174806f750 0x7f17481626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.380+0000 7f1747e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17480aebd0 con 0x7f17481478b0 2024-04-15T15:25:52.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.380+0000 7f1747e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f174806fbb0 con 0x7f1748147c90 2024-04-15T15:25:52.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.380+0000 7f1747e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f17480aee50 con 0x7f174806f750 2024-04-15T15:25:52.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.380+0000 7f1746a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f17481478b0 0x7f17480afbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:53712/0 (socket says 172.21.15.150:53712) 2024-04-15T15:25:52.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.380+0000 7f1746a00640 1 -- 172.21.15.150:0/190449620 learned_addr learned my addr 172.21.15.150:0/190449620 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:52.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.380+0000 7f1746000640 1 --2- 172.21.15.150:0/190449620 >> v2:172.21.15.150:3300/0 conn(0x7f1748147c90 0x7f174806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.381+0000 7f1747400640 1 --2- 172.21.15.150:0/190449620 >> v2:172.21.15.187:3300/0 conn(0x7f174806f750 0x7f17481626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.381+0000 7f1745600640 1 -- 172.21.15.150:0/190449620 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278287192 0 0) 0x7f1738001110 con 0x7f17481478b0 2024-04-15T15:25:52.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.381+0000 7f1745600640 1 -- 172.21.15.150:0/190449620 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f172c003520 con 0x7f17481478b0 2024-04-15T15:25:52.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.381+0000 7f1746000640 1 -- 172.21.15.150:0/190449620 >> v2:172.21.15.187:3300/0 conn(0x7f174806f750 msgr2=0x7f17481626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:52.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.381+0000 7f1746000640 1 --2- 172.21.15.150:0/190449620 >> v2:172.21.15.187:3300/0 conn(0x7f174806f750 0x7f17481626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.381+0000 7f1746000640 1 -- 172.21.15.150:0/190449620 >> v1:172.21.15.150:6789/0 conn(0x7f17481478b0 legacy=0x7f17480afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.381+0000 7f1746000640 1 -- 172.21.15.150:0/190449620 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17480af110 con 0x7f1748147c90 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.382+0000 7f1746000640 1 --2- 172.21.15.150:0/190449620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1748147c90 0x7f174806f210 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f174806c860 tx=0x7f173402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470c1233b6ceecdc server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.382+0000 7f1745600640 1 -- 172.21.15.150:0/190449620 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734035020 con 0x7f1748147c90 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.382+0000 7f1745600640 1 -- 172.21.15.150:0/190449620 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1734031830 con 0x7f1748147c90 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.382+0000 7f1745600640 1 -- 172.21.15.150:0/190449620 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734031be0 con 0x7f1748147c90 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.383+0000 7f1747e00640 1 -- 172.21.15.150:0/190449620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1748147c90 msgr2=0x7f174806f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.383+0000 7f1747e00640 1 --2- 172.21.15.150:0/190449620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1748147c90 0x7f174806f210 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f174806c860 tx=0x7f173402b8d0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.383+0000 7f1747e00640 1 -- 172.21.15.150:0/190449620 shutdown_connections 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.383+0000 7f1747e00640 1 --2- 172.21.15.150:0/190449620 >> v2:172.21.15.187:3300/0 conn(0x7f174806f750 0x7f17481626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.383+0000 7f1747e00640 1 --2- 172.21.15.150:0/190449620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1748147c90 0x7f174806f210 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.383+0000 7f1747e00640 1 -- 172.21.15.150:0/190449620 >> 172.21.15.150:0/190449620 conn(0x7f174806b930 msgr2=0x7f174806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:52.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.383+0000 7f1747e00640 1 -- 172.21.15.150:0/190449620 shutdown_connections 2024-04-15T15:25:52.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.383+0000 7f1747e00640 1 -- 172.21.15.150:0/190449620 wait complete. 2024-04-15T15:25:52.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.385+0000 7f1747e00640 1 Processor -- start 2024-04-15T15:25:52.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.385+0000 7f1747e00640 1 -- start start 2024-04-15T15:25:52.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1747e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f174806f750 0x7f174815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1747e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17481478b0 0x7f174815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1747e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1748147c90 0x7f17480ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1747e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1748162c00 con 0x7f174806f750 2024-04-15T15:25:52.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1747e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17480af6d0 con 0x7f17481478b0 2024-04-15T15:25:52.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1747e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1748162d80 con 0x7f1748147c90 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1746a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f174806f750 0x7f174815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1746a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f174806f750 0x7f174815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49678/0 (socket says 172.21.15.150:49678) 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.386+0000 7f1746a00640 1 -- 172.21.15.150:0/3405748094 learned_addr learned my addr 172.21.15.150:0/3405748094 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f1746a00640 1 -- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1748147c90 msgr2=0x7f17480ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f1746a00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1748147c90 0x7f17480ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f1746a00640 1 -- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17481478b0 msgr2=0x7f174815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f1746a00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17481478b0 0x7f174815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f1746a00640 1 -- 172.21.15.150:0/3405748094 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174815f660 con 0x7f174806f750 2024-04-15T15:25:52.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f1746a00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f174806f750 0x7f174815e0d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f173800c730 tx=0x7f173800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:52.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f173ea00640 1 -- 172.21.15.150:0/3405748094 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738017070 con 0x7f174806f750 2024-04-15T15:25:52.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f173ea00640 1 -- 172.21.15.150:0/3405748094 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1738005ad0 con 0x7f174806f750 2024-04-15T15:25:52.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174815f940 con 0x7f174806f750 2024-04-15T15:25:52.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f173ea00640 1 -- 172.21.15.150:0/3405748094 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738007e20 con 0x7f174806f750 2024-04-15T15:25:52.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.387+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17481601e0 con 0x7f174806f750 2024-04-15T15:25:52.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.389+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174815fd30 con 0x7f174806f750 2024-04-15T15:25:52.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.389+0000 7f173ea00640 1 -- 172.21.15.150:0/3405748094 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f173801a070 con 0x7f174806f750 2024-04-15T15:25:52.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.391+0000 7f173ea00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f172806c980 0x7f172806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.391+0000 7f173ea00640 1 -- 172.21.15.150:0/3405748094 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f173808b0b0 con 0x7f174806f750 2024-04-15T15:25:52.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.392+0000 7f1746000640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f172806c980 0x7f172806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.393+0000 7f1746000640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f172806c980 0x7f172806ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1734036000 tx=0x7f1734030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:52.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.395+0000 7f173ea00640 1 -- 172.21.15.150:0/3405748094 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17480aebd0 con 0x7f174806f750 2024-04-15T15:25:52.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.592+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f174806c8c0 con 0x7f174806f750 2024-04-15T15:25:52.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.593+0000 7f173ea00640 1 -- 172.21.15.150:0/3405748094 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f1738059390 con 0x7f174806f750 2024-04-15T15:25:52.595 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-04-15T15:25:52.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f172806c980 msgr2=0x7f172806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f172806c980 0x7f172806ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1734036000 tx=0x7f1734030000 comp rx=0 tx=0).stop 2024-04-15T15:25:52.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f174806f750 msgr2=0x7f174815e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f174806f750 0x7f174815e0d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f173800c730 tx=0x7f173800cbf0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 shutdown_connections 2024-04-15T15:25:52.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f172806c980 0x7f172806ee40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1748147c90 0x7f17480ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17481478b0 0x7f174815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 --2- 172.21.15.150:0/3405748094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f174806f750 0x7f174815e0d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 >> 172.21.15.150:0/3405748094 conn(0x7f174806b930 msgr2=0x7f1748146950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:52.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 shutdown_connections 2024-04-15T15:25:52.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.596+0000 7f1747e00640 1 -- 172.21.15.150:0/3405748094 wait complete. 2024-04-15T15:25:52.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:52.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.819+0000 7ff58a600640 1 Processor -- start 2024-04-15T15:25:52.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.819+0000 7ff58a600640 1 -- start start 2024-04-15T15:25:52.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.821+0000 7ff58a600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5841461f0 0x7ff5840ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.821+0000 7ff58a600640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5840b4be0 con 0x7ff5840b3fe0 2024-04-15T15:25:52.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.821+0000 7ff58a600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5840b4620 con 0x7ff5840cedf0 2024-04-15T15:25:52.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.821+0000 7ff58a600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5840a9640 con 0x7ff5841461f0 2024-04-15T15:25:52.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.821+0000 7ff583e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7ff5840b3fe0 0x7ff58415d7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:53720/0 (socket says 172.21.15.150:53720) 2024-04-15T15:25:52.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.821+0000 7ff583e00640 1 -- 172.21.15.150:0/4022746919 learned_addr learned my addr 172.21.15.150:0/4022746919 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.821+0000 7ff583400640 1 --2- 172.21.15.150:0/4022746919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5841461f0 0x7ff5840ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.822+0000 7ff582000640 1 -- 172.21.15.150:0/4022746919 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993188363 0 0) 0x7ff5840b4be0 con 0x7ff5840b3fe0 2024-04-15T15:25:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.822+0000 7ff582000640 1 -- 172.21.15.150:0/4022746919 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff568003520 con 0x7ff5840b3fe0 2024-04-15T15:25:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.822+0000 7ff582000640 1 -- 172.21.15.150:0/4022746919 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1872330037 0 0) 0x7ff568003520 con 0x7ff5840b3fe0 2024-04-15T15:25:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.822+0000 7ff583400640 1 -- 172.21.15.150:0/4022746919 >> v1:172.21.15.187:6789/0 conn(0x7ff5840cedf0 legacy=0x7ff5840b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.822+0000 7ff583400640 1 -- 172.21.15.150:0/4022746919 >> v1:172.21.15.150:6789/0 conn(0x7ff5840b3fe0 legacy=0x7ff58415d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.822+0000 7ff583400640 1 -- 172.21.15.150:0/4022746919 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5840b4900 con 0x7ff5841461f0 2024-04-15T15:25:52.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.824+0000 7ff583400640 1 --2- 172.21.15.150:0/4022746919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5841461f0 0x7ff5840ce8b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff574009800 tx=0x7ff57402b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4553c0c065d94535 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.824+0000 7ff582000640 1 -- 172.21.15.150:0/4022746919 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574037070 con 0x7ff5841461f0 2024-04-15T15:25:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.824+0000 7ff582000640 1 -- 172.21.15.150:0/4022746919 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff574030510 con 0x7ff5841461f0 2024-04-15T15:25:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.824+0000 7ff582000640 1 -- 172.21.15.150:0/4022746919 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0019f0 con 0x7ff5841461f0 2024-04-15T15:25:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.825+0000 7ff58a600640 1 -- 172.21.15.150:0/4022746919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5841461f0 msgr2=0x7ff5840ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.825+0000 7ff58a600640 1 --2- 172.21.15.150:0/4022746919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5841461f0 0x7ff5840ce8b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff574009800 tx=0x7ff57402b690 comp rx=0 tx=0).stop 2024-04-15T15:25:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.825+0000 7ff58a600640 1 -- 172.21.15.150:0/4022746919 shutdown_connections 2024-04-15T15:25:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.825+0000 7ff58a600640 1 --2- 172.21.15.150:0/4022746919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5841461f0 0x7ff5840ce8b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.825+0000 7ff58a600640 1 -- 172.21.15.150:0/4022746919 >> 172.21.15.150:0/4022746919 conn(0x7ff5840b27b0 msgr2=0x7ff5840b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.825+0000 7ff58a600640 1 -- 172.21.15.150:0/4022746919 shutdown_connections 2024-04-15T15:25:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.825+0000 7ff58a600640 1 -- 172.21.15.150:0/4022746919 wait complete. 2024-04-15T15:25:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.827+0000 7ff58a600640 1 Processor -- start 2024-04-15T15:25:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.827+0000 7ff58a600640 1 -- start start 2024-04-15T15:25:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff58a600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5840b3fe0 0x7ff58406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff58a600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5840cedf0 0x7ff58413a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff58a600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5841461f0 0x7ff58413fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff58a600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff58415d290 con 0x7ff5841461f0 2024-04-15T15:25:52.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff58a600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5841465d0 con 0x7ff5840cedf0 2024-04-15T15:25:52.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff58a600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff58415dec0 con 0x7ff5840b3fe0 2024-04-15T15:25:52.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff583e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5841461f0 0x7ff58413fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff583e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5841461f0 0x7ff58413fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49682/0 (socket says 172.21.15.150:49682) 2024-04-15T15:25:52.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff583e00640 1 -- 172.21.15.150:0/561776835 learned_addr learned my addr 172.21.15.150:0/561776835 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff582a00640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5840cedf0 0x7ff58413a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.828+0000 7ff583400640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5840b3fe0 0x7ff58406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff583e00640 1 -- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5840b3fe0 msgr2=0x7ff58406f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff583e00640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5840b3fe0 0x7ff58406f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff583e00640 1 -- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5840cedf0 msgr2=0x7ff58413a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff583e00640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5840cedf0 0x7ff58413a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff583e00640 1 -- 172.21.15.150:0/561776835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58413ae10 con 0x7ff5841461f0 2024-04-15T15:25:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff583e00640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5841461f0 0x7ff58413fe00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00caa0 tx=0x7ff57c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff57b400640 1 -- 172.21.15.150:0/561776835 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c018070 con 0x7ff5841461f0 2024-04-15T15:25:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff57b400640 1 -- 172.21.15.150:0/561776835 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c00a6d0 con 0x7ff5841461f0 2024-04-15T15:25:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff584140340 con 0x7ff5841461f0 2024-04-15T15:25:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff57b400640 1 -- 172.21.15.150:0/561776835 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00aac0 con 0x7ff5841461f0 2024-04-15T15:25:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.829+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5841408f0 con 0x7ff5841461f0 2024-04-15T15:25:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.831+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff584068c80 con 0x7ff5841461f0 2024-04-15T15:25:52.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.831+0000 7ff57b400640 1 -- 172.21.15.150:0/561776835 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff57c01b070 con 0x7ff5841461f0 2024-04-15T15:25:52.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.832+0000 7ff57b400640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff56406c980 0x7ff56406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:52.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.832+0000 7ff57b400640 1 -- 172.21.15.150:0/561776835 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7ff57c08b4b0 con 0x7ff5841461f0 2024-04-15T15:25:52.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.833+0000 7ff583400640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff56406c980 0x7ff56406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:52.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.835+0000 7ff583400640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff56406c980 0x7ff56406ee40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff574034000 tx=0x7ff574008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:52.837+0000 7ff57b400640 1 -- 172.21.15.150:0/561776835 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff57c059790 con 0x7ff5841461f0 2024-04-15T15:25:53.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.077+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff58413afe0 con 0x7ff5841461f0 2024-04-15T15:25:53.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.077+0000 7ff57b400640 1 -- 172.21.15.150:0/561776835 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5194 (secure 0 0 0) 0x7ff57c059090 con 0x7ff5841461f0 2024-04-15T15:25:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:53.079 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:37.899562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5499":{"gid":5499,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6835/3243584315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3243584315},{"type":"v1","addr":"172.21.15.150:6835","nonce":3243584315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5499,"qdb_cluster":[5499]},"id":2}]} 2024-04-15T15:25:53.080 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 22 2024-04-15T15:25:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.080+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff56406c980 msgr2=0x7ff56406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.080+0000 7ff58a600640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff56406c980 0x7ff56406ee40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff574034000 tx=0x7ff574008030 comp rx=0 tx=0).stop 2024-04-15T15:25:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.080+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5841461f0 msgr2=0x7ff58413fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.080+0000 7ff58a600640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5841461f0 0x7ff58413fe00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00caa0 tx=0x7ff57c00cf60 comp rx=0 tx=0).stop 2024-04-15T15:25:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.081+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 shutdown_connections 2024-04-15T15:25:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.081+0000 7ff58a600640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5841461f0 0x7ff58413fe00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.081+0000 7ff58a600640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5840cedf0 0x7ff58413a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.081+0000 7ff58a600640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7ff56406c980 0x7ff56406ee40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.081+0000 7ff58a600640 1 --2- 172.21.15.150:0/561776835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5840b3fe0 0x7ff58406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.081+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 >> 172.21.15.150:0/561776835 conn(0x7ff5840b27b0 msgr2=0x7ff584141350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.081+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 shutdown_connections 2024-04-15T15:25:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.081+0000 7ff58a600640 1 -- 172.21.15.150:0/561776835 wait complete. 2024-04-15T15:25:53.102 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:53.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.310+0000 7f4343200640 1 Processor -- start 2024-04-15T15:25:53.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.310+0000 7f4343200640 1 -- start start 2024-04-15T15:25:53.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.312+0000 7f4343200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f43440d1c90 0x7f4344162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.312+0000 7f4343200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43440cf110 0x7f43440d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.312+0000 7f4343200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43440d0090 con 0x7f4344162020 2024-04-15T15:25:53.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.312+0000 7f4343200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4344163440 con 0x7f43440d1c90 2024-04-15T15:25:53.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.312+0000 7f4343200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43440b7b30 con 0x7f43440cf110 2024-04-15T15:25:53.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.312+0000 7f4342800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43440cf110 0x7f43440d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.313+0000 7f4342800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43440cf110 0x7f43440d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:39670/0 (socket says 172.21.15.150:39670) 2024-04-15T15:25:53.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.313+0000 7f4342800640 1 -- 172.21.15.150:0/1482118962 learned_addr learned my addr 172.21.15.150:0/1482118962 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:53.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.313+0000 7f4339400640 1 --2- 172.21.15.150:0/1482118962 >> v2:172.21.15.187:3300/0 conn(0x7f43440d1c90 0x7f4344162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.313+0000 7f4342800640 1 -- 172.21.15.150:0/1482118962 >> v2:172.21.15.187:3300/0 conn(0x7f43440d1c90 msgr2=0x7f4344162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:53.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.313+0000 7f4342800640 1 --2- 172.21.15.150:0/1482118962 >> v2:172.21.15.187:3300/0 conn(0x7f43440d1c90 0x7f4344162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.313+0000 7f4342800640 1 -- 172.21.15.150:0/1482118962 >> v1:172.21.15.187:6789/0 conn(0x7f4344162020 legacy=0x7f43440d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.313+0000 7f4342800640 1 -- 172.21.15.150:0/1482118962 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43440cfad0 con 0x7f43440cf110 2024-04-15T15:25:53.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.315+0000 7f4342800640 1 --2- 172.21.15.150:0/1482118962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43440cf110 0x7f43440d3390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f433c009e50 tx=0x7f433c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33d1e80d64a3d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:53.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.315+0000 7f4341400640 1 -- 172.21.15.150:0/1482118962 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c02cb00 con 0x7f43440cf110 2024-04-15T15:25:53.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.315+0000 7f4341400640 1 -- 172.21.15.150:0/1482118962 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433c02cca0 con 0x7f43440cf110 2024-04-15T15:25:53.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.315+0000 7f4341400640 1 -- 172.21.15.150:0/1482118962 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c034610 con 0x7f43440cf110 2024-04-15T15:25:53.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.316+0000 7f4343200640 1 -- 172.21.15.150:0/1482118962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43440cf110 msgr2=0x7f43440d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.316+0000 7f4343200640 1 --2- 172.21.15.150:0/1482118962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43440cf110 0x7f43440d3390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f433c009e50 tx=0x7f433c02bf60 comp rx=0 tx=0).stop 2024-04-15T15:25:53.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.316+0000 7f4343200640 1 -- 172.21.15.150:0/1482118962 shutdown_connections 2024-04-15T15:25:53.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.316+0000 7f4343200640 1 --2- 172.21.15.150:0/1482118962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43440cf110 0x7f43440d3390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.316+0000 7f4343200640 1 --2- 172.21.15.150:0/1482118962 >> v2:172.21.15.187:3300/0 conn(0x7f43440d1c90 0x7f4344162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.316+0000 7f4343200640 1 -- 172.21.15.150:0/1482118962 >> 172.21.15.150:0/1482118962 conn(0x7f434406cf00 msgr2=0x7f434406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:53.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.316+0000 7f4343200640 1 -- 172.21.15.150:0/1482118962 shutdown_connections 2024-04-15T15:25:53.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.316+0000 7f4343200640 1 -- 172.21.15.150:0/1482118962 wait complete. 2024-04-15T15:25:53.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.318+0000 7f4343200640 1 Processor -- start 2024-04-15T15:25:53.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.318+0000 7f4343200640 1 -- start start 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4343200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440cf110 0x7f434415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4343200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43440d1c90 0x7f43440c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4343200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4344162020 0x7f434415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4343200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43440d2b90 con 0x7f43440cf110 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4343200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43440cf880 con 0x7f43440d1c90 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4343200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43440d2e70 con 0x7f4344162020 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4341e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440cf110 0x7f434415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4341e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440cf110 0x7f434415b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49690/0 (socket says 172.21.15.150:49690) 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4341e00640 1 -- 172.21.15.150:0/2515908412 learned_addr learned my addr 172.21.15.150:0/2515908412 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:53.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4342800640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4344162020 0x7f434415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4339400640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43440d1c90 0x7f43440c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4341e00640 1 -- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4344162020 msgr2=0x7f434415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4341e00640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4344162020 0x7f434415e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4341e00640 1 -- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43440d1c90 msgr2=0x7f43440c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4341e00640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43440d1c90 0x7f43440c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.319+0000 7f4341e00640 1 -- 172.21.15.150:0/2515908412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43440ccba0 con 0x7f43440cf110 2024-04-15T15:25:53.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.320+0000 7f4339400640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43440d1c90 0x7f43440c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:53.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.320+0000 7f4341e00640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440cf110 0x7f434415b460 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f433400c450 tx=0x7f433400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:53.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.320+0000 7f433aa00640 1 -- 172.21.15.150:0/2515908412 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340052d0 con 0x7f43440cf110 2024-04-15T15:25:53.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.320+0000 7f433aa00640 1 -- 172.21.15.150:0/2515908412 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4334005470 con 0x7f43440cf110 2024-04-15T15:25:53.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.320+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43440cce80 con 0x7f43440cf110 2024-04-15T15:25:53.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.320+0000 7f433aa00640 1 -- 172.21.15.150:0/2515908412 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334012870 con 0x7f43440cf110 2024-04-15T15:25:53.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.320+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43440cd440 con 0x7f43440cf110 2024-04-15T15:25:53.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.321+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43440755c0 con 0x7f43440cf110 2024-04-15T15:25:53.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.322+0000 7f433aa00640 1 -- 172.21.15.150:0/2515908412 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4334012a10 con 0x7f43440cf110 2024-04-15T15:25:53.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.323+0000 7f433aa00640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f432006c9b0 0x7f432006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.323+0000 7f433aa00640 1 -- 172.21.15.150:0/2515908412 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f433408b910 con 0x7f43440cf110 2024-04-15T15:25:53.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.324+0000 7f4339400640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f432006c9b0 0x7f432006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.325+0000 7f4339400640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f432006c9b0 0x7f432006ee70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f434406d8b0 tx=0x7f432c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:53.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.327+0000 7f433aa00640 1 -- 172.21.15.150:0/2515908412 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43440d0090 con 0x7f43440cf110 2024-04-15T15:25:53.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.529+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43440b7650 con 0x7f43440cf110 2024-04-15T15:25:53.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.531+0000 7f433aa00640 1 -- 172.21.15.150:0/2515908412 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16848 (secure 0 0 0) 0x7f4334059bf0 con 0x7f43440cf110 2024-04-15T15:25:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:53.533 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":36,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:33.990388+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/3327940880":"2024-04-16T15:25:33.990360+0000","172.21.15.150:6833/3327940880":"2024-04-16T15:25:33.990360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.534+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f432006c9b0 msgr2=0x7f432006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.534+0000 7f4343200640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f432006c9b0 0x7f432006ee70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f434406d8b0 tx=0x7f432c008040 comp rx=0 tx=0).stop 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440cf110 msgr2=0x7f434415b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440cf110 0x7f434415b460 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f433400c450 tx=0x7f433400c910 comp rx=0 tx=0).stop 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 shutdown_connections 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f432006c9b0 0x7f432006ee70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4344162020 0x7f434415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43440d1c90 0x7f43440c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 --2- 172.21.15.150:0/2515908412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440cf110 0x7f434415b460 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 >> 172.21.15.150:0/2515908412 conn(0x7f434406cf00 msgr2=0x7f434415ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:53.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 shutdown_connections 2024-04-15T15:25:53.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.535+0000 7f4343200640 1 -- 172.21.15.150:0/2515908412 wait complete. 2024-04-15T15:25:53.555 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:25:53.555 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:53.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.749+0000 7f038e000640 1 Processor -- start 2024-04-15T15:25:53.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.749+0000 7f038e000640 1 -- start start 2024-04-15T15:25:53.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.751+0000 7f038e000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0388155320 0x7f038815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.751+0000 7f038e000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0388072ef0 0x7f03880cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.751+0000 7f038e000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03880b5320 con 0x7f038806ea00 2024-04-15T15:25:53.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.751+0000 7f038e000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0388155700 con 0x7f0388072ef0 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.751+0000 7f038e000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f038806dcd0 con 0x7f0388155320 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.751+0000 7f038d600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0388072ef0 0x7f03880cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.752+0000 7f038d600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0388072ef0 0x7f03880cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49702/0 (socket says 172.21.15.150:49702) 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.752+0000 7f038d600640 1 -- 172.21.15.150:0/2656980396 learned_addr learned my addr 172.21.15.150:0/2656980396 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.752+0000 7f038cc00640 1 --2- 172.21.15.150:0/2656980396 >> v2:172.21.15.187:3300/0 conn(0x7f0388155320 0x7f038815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.752+0000 7f038d600640 1 -- 172.21.15.150:0/2656980396 >> v2:172.21.15.187:3300/0 conn(0x7f0388155320 msgr2=0x7f038815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.752+0000 7f038d600640 1 --2- 172.21.15.150:0/2656980396 >> v2:172.21.15.187:3300/0 conn(0x7f0388155320 0x7f038815bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.752+0000 7f038d600640 1 -- 172.21.15.150:0/2656980396 >> v1:172.21.15.187:6789/0 conn(0x7f038806ea00 legacy=0x7f03880727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:53.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.752+0000 7f038d600640 1 -- 172.21.15.150:0/2656980396 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038806df20 con 0x7f0388072ef0 2024-04-15T15:25:53.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.753+0000 7f038d600640 1 --2- 172.21.15.150:0/2656980396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f03880cebc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f03880b5d70 tx=0x7f038002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a3eea30483bf56 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:53.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.753+0000 7f0387400640 1 -- 172.21.15.150:0/2656980396 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038002e070 con 0x7f0388072ef0 2024-04-15T15:25:53.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.753+0000 7f0387400640 1 -- 172.21.15.150:0/2656980396 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0380037070 con 0x7f0388072ef0 2024-04-15T15:25:53.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.753+0000 7f0387400640 1 -- 172.21.15.150:0/2656980396 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380031a00 con 0x7f0388072ef0 2024-04-15T15:25:53.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.754+0000 7f038e000640 1 -- 172.21.15.150:0/2656980396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 msgr2=0x7f03880cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.754+0000 7f038e000640 1 --2- 172.21.15.150:0/2656980396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f03880cebc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f03880b5d70 tx=0x7f038002b880 comp rx=0 tx=0).stop 2024-04-15T15:25:53.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.754+0000 7f038e000640 1 -- 172.21.15.150:0/2656980396 shutdown_connections 2024-04-15T15:25:53.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.754+0000 7f038e000640 1 --2- 172.21.15.150:0/2656980396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f03880cebc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.754+0000 7f038e000640 1 --2- 172.21.15.150:0/2656980396 >> v2:172.21.15.187:3300/0 conn(0x7f0388155320 0x7f038815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.754+0000 7f038e000640 1 -- 172.21.15.150:0/2656980396 >> 172.21.15.150:0/2656980396 conn(0x7f038806b930 msgr2=0x7f038806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:53.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.754+0000 7f038e000640 1 -- 172.21.15.150:0/2656980396 shutdown_connections 2024-04-15T15:25:53.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.754+0000 7f038e000640 1 -- 172.21.15.150:0/2656980396 wait complete. 2024-04-15T15:25:53.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.756+0000 7f038e000640 1 Processor -- start 2024-04-15T15:25:53.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.756+0000 7f038e000640 1 -- start start 2024-04-15T15:25:53.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f038e000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f038806ea00 0x7f038814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f038e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f038815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f038e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0388155320 0x7f038815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f038e000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f03880b4d60 con 0x7f0388072ef0 2024-04-15T15:25:53.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f038e000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f038806e4e0 con 0x7f038806ea00 2024-04-15T15:25:53.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f038e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03880b4ee0 con 0x7f0388155320 2024-04-15T15:25:53.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f0387e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f038815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f0387e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f038815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49704/0 (socket says 172.21.15.150:49704) 2024-04-15T15:25:53.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f0387e00640 1 -- 172.21.15.150:0/4072122314 learned_addr learned my addr 172.21.15.150:0/4072122314 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:53.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f0387e00640 1 -- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0388155320 msgr2=0x7f038815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f0387e00640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0388155320 0x7f038815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f0387e00640 1 -- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f038806ea00 msgr2=0x7f038814d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:53.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.757+0000 7f0387e00640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f038806ea00 0x7f038814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:53.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.758+0000 7f0387e00640 1 -- 172.21.15.150:0/4072122314 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03880ae360 con 0x7f0388072ef0 2024-04-15T15:25:53.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.758+0000 7f0387e00640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f038815d3c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f037800c7d0 tx=0x7f037800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:53.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.758+0000 7f0384c00640 1 -- 172.21.15.150:0/4072122314 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378017070 con 0x7f0388072ef0 2024-04-15T15:25:53.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.758+0000 7f0384c00640 1 -- 172.21.15.150:0/4072122314 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0378003da0 con 0x7f0388072ef0 2024-04-15T15:25:53.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.758+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03880af1a0 con 0x7f0388072ef0 2024-04-15T15:25:53.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.758+0000 7f0384c00640 1 -- 172.21.15.150:0/4072122314 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03780052b0 con 0x7f0388072ef0 2024-04-15T15:25:53.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.758+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03880af760 con 0x7f0388072ef0 2024-04-15T15:25:53.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.759+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03880755c0 con 0x7f0388072ef0 2024-04-15T15:25:53.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.760+0000 7f0384c00640 1 -- 172.21.15.150:0/4072122314 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f037801a070 con 0x7f0388072ef0 2024-04-15T15:25:53.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.761+0000 7f0384c00640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f036406c980 0x7f036406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:53.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.761+0000 7f0384c00640 1 -- 172.21.15.150:0/4072122314 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f037808b3c0 con 0x7f0388072ef0 2024-04-15T15:25:53.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.762+0000 7f038cc00640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f036406c980 0x7f036406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:53.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.763+0000 7f038cc00640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f036406c980 0x7f036406ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f037000a810 tx=0x7f0370008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:53.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.766+0000 7f0384c00640 1 -- 172.21.15.150:0/4072122314 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0378059710 con 0x7f0388072ef0 2024-04-15T15:25:53.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.993+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0388068a60 con 0x7f0388072ef0 2024-04-15T15:25:53.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.995+0000 7f0384c00640 1 -- 172.21.15.150:0/4072122314 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5194 (secure 0 0 0) 0x7f0378059010 con 0x7f0388072ef0 2024-04-15T15:25:53.996 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:53.996 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:37.899562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5499":{"gid":5499,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6835/3243584315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3243584315},{"type":"v1","addr":"172.21.15.150:6835","nonce":3243584315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5499,"qdb_cluster":[5499]},"id":2}]} 2024-04-15T15:25:53.997 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 22 2024-04-15T15:25:54.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.997+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f036406c980 msgr2=0x7f036406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.997+0000 7f038e000640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f036406c980 0x7f036406ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f037000a810 tx=0x7f0370008040 comp rx=0 tx=0).stop 2024-04-15T15:25:54.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.997+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 msgr2=0x7f038815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.997+0000 7f038e000640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f038815d3c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f037800c7d0 tx=0x7f037800cc90 comp rx=0 tx=0).stop 2024-04-15T15:25:54.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.998+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 shutdown_connections 2024-04-15T15:25:54.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.998+0000 7f038e000640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f036406c980 0x7f036406ee40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.998+0000 7f038e000640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0388155320 0x7f038815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.998+0000 7f038e000640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0388072ef0 0x7f038815d3c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.998+0000 7f038e000640 1 --2- 172.21.15.150:0/4072122314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f038806ea00 0x7f038814d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.998+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 >> 172.21.15.150:0/4072122314 conn(0x7f038806b930 msgr2=0x7f038806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:54.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.998+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 shutdown_connections 2024-04-15T15:25:54.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:53.998+0000 7f038e000640 1 -- 172.21.15.150:0/4072122314 wait complete. 2024-04-15T15:25:54.017 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:25:54.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.224+0000 7f99ca800640 1 Processor -- start 2024-04-15T15:25:54.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.225+0000 7f99ca800640 1 -- start start 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.226+0000 7f99ca800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f99c41461f0 0x7f99c40ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.226+0000 7f99ca800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c40b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.226+0000 7f99ca800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c40b4be0 con 0x7f99c40b3e10 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.226+0000 7f99ca800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f99c40a9640 con 0x7f99c41461f0 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.226+0000 7f99ca800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f99c40cf250 con 0x7f99c40cedf0 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.226+0000 7f99c3400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f99c41461f0 0x7f99c40ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.227+0000 7f99c3400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f99c41461f0 0x7f99c40ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49720/0 (socket says 172.21.15.150:49720) 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.227+0000 7f99c3400640 1 -- 172.21.15.150:0/3914034857 learned_addr learned my addr 172.21.15.150:0/3914034857 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:54.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.227+0000 7f99baa00640 1 --2- 172.21.15.150:0/3914034857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c40b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.227+0000 7f99c3400640 1 -- 172.21.15.150:0/3914034857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 msgr2=0x7f99c40b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:54.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.227+0000 7f99c3400640 1 --2- 172.21.15.150:0/3914034857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c40b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.227+0000 7f99c3400640 1 -- 172.21.15.150:0/3914034857 >> v1:172.21.15.187:6789/0 conn(0x7f99c40b3e10 legacy=0x7f99c40cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:54.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.227+0000 7f99c3400640 1 -- 172.21.15.150:0/3914034857 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c40b4620 con 0x7f99c41461f0 2024-04-15T15:25:54.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.227+0000 7f99baa00640 1 --2- 172.21.15.150:0/3914034857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c40b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:54.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.228+0000 7f99c3400640 1 --2- 172.21.15.150:0/3914034857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ce8b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098a0 tx=0x7f99b402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370a924ed1ed7398 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:54.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.228+0000 7f99c2a00640 1 -- 172.21.15.150:0/3914034857 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4035020 con 0x7f99c41461f0 2024-04-15T15:25:54.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.228+0000 7f99c2a00640 1 -- 172.21.15.150:0/3914034857 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b4031640 con 0x7f99c41461f0 2024-04-15T15:25:54.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.228+0000 7f99c2a00640 1 -- 172.21.15.150:0/3914034857 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b402d040 con 0x7f99c41461f0 2024-04-15T15:25:54.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.229+0000 7f99ca800640 1 -- 172.21.15.150:0/3914034857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 msgr2=0x7f99c40ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.229+0000 7f99ca800640 1 --2- 172.21.15.150:0/3914034857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ce8b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098a0 tx=0x7f99b402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.229+0000 7f99ca800640 1 -- 172.21.15.150:0/3914034857 shutdown_connections 2024-04-15T15:25:54.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.229+0000 7f99ca800640 1 --2- 172.21.15.150:0/3914034857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c40b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.229+0000 7f99ca800640 1 --2- 172.21.15.150:0/3914034857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ce8b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.229+0000 7f99ca800640 1 -- 172.21.15.150:0/3914034857 >> 172.21.15.150:0/3914034857 conn(0x7f99c40b27b0 msgr2=0x7f99c40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:54.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.229+0000 7f99ca800640 1 -- 172.21.15.150:0/3914034857 shutdown_connections 2024-04-15T15:25:54.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.229+0000 7f99ca800640 1 -- 172.21.15.150:0/3914034857 wait complete. 2024-04-15T15:25:54.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.231+0000 7f99ca800640 1 Processor -- start 2024-04-15T15:25:54.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.231+0000 7f99ca800640 1 -- start start 2024-04-15T15:25:54.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99ca800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99c40b3e10 0x7f99c40abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99ca800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99ca800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99ca800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99c415d290 con 0x7f99c41461f0 2024-04-15T15:25:54.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99ca800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99c415cfb0 con 0x7f99c40b3e10 2024-04-15T15:25:54.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99ca800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f99c41465d0 con 0x7f99c40cedf0 2024-04-15T15:25:54.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99c3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99c3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49732/0 (socket says 172.21.15.150:49732) 2024-04-15T15:25:54.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.232+0000 7f99c3e00640 1 -- 172.21.15.150:0/1094159285 learned_addr learned my addr 172.21.15.150:0/1094159285 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:54.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99baa00640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99c3400640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99c40b3e10 0x7f99c40abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99c3e00640 1 -- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 msgr2=0x7f99c415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99c3e00640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99c3e00640 1 -- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99c40b3e10 msgr2=0x7f99c40abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99c3e00640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99c40b3e10 0x7f99c40abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99c3e00640 1 -- 172.21.15.150:0/1094159285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c40ac190 con 0x7f99c41461f0 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99baa00640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99c3400640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99c40b3e10 0x7f99c40abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99c3e00640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ae5d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00c780 tx=0x7f99bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99bbe00640 1 -- 172.21.15.150:0/1094159285 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc004b60 con 0x7f99c41461f0 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99bbe00640 1 -- 172.21.15.150:0/1094159285 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99bc004d00 con 0x7f99c41461f0 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99bbe00640 1 -- 172.21.15.150:0/1094159285 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00aa90 con 0x7f99c41461f0 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.233+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c40aa520 con 0x7f99c41461f0 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.234+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c40aaae0 con 0x7f99c41461f0 2024-04-15T15:25:54.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.235+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c4068c80 con 0x7f99c41461f0 2024-04-15T15:25:54.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.236+0000 7f99bbe00640 1 -- 172.21.15.150:0/1094159285 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f99bc00ac30 con 0x7f99c41461f0 2024-04-15T15:25:54.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.237+0000 7f99bbe00640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f99a006c980 0x7f99a006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.237+0000 7f99bbe00640 1 -- 172.21.15.150:0/1094159285 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f99bc08bfc0 con 0x7f99c41461f0 2024-04-15T15:25:54.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.237+0000 7f99c3400640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f99a006c980 0x7f99a006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.238+0000 7f99c3400640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f99a006c980 0x7f99a006ee40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99b4036000 tx=0x7f99b4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.241+0000 7f99bbe00640 1 -- 172.21.15.150:0/1094159285 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99bc020070 con 0x7f99c41461f0 2024-04-15T15:25:54.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.464+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99c40b2070 con 0x7f99c41461f0 2024-04-15T15:25:54.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.465+0000 7f99bbe00640 1 -- 172.21.15.150:0/1094159285 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5194 (secure 0 0 0) 0x7f99bc05a2a0 con 0x7f99c41461f0 2024-04-15T15:25:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:54.467 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-15T15:25:06.343634+0000","modified":"2024-04-15T15:25:37.899562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5499":{"gid":5499,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6835/3243584315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3243584315},{"type":"v1","addr":"172.21.15.150:6835","nonce":3243584315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5499,"qdb_cluster":[5499]},"id":2}]} 2024-04-15T15:25:54.467 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 22 2024-04-15T15:25:54.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.467+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f99a006c980 msgr2=0x7f99a006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.467+0000 7f99ca800640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f99a006c980 0x7f99a006ee40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99b4036000 tx=0x7f99b4008030 comp rx=0 tx=0).stop 2024-04-15T15:25:54.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.467+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 msgr2=0x7f99c40ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.467+0000 7f99ca800640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ae5d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00c780 tx=0x7f99bc00cc40 comp rx=0 tx=0).stop 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.467+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 shutdown_connections 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.467+0000 7f99ca800640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c41461f0 0x7f99c40ae5d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.467+0000 7f99ca800640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f99a006c980 0x7f99a006ee40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.468+0000 7f99ca800640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99c40cedf0 0x7f99c415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.468+0000 7f99ca800640 1 --2- 172.21.15.150:0/1094159285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99c40b3e10 0x7f99c40abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.468+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 >> 172.21.15.150:0/1094159285 conn(0x7f99c40b27b0 msgr2=0x7f99c40b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.468+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 shutdown_connections 2024-04-15T15:25:54.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.468+0000 7f99ca800640 1 -- 172.21.15.150:0/1094159285 wait complete. 2024-04-15T15:25:54.489 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:25:54.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.696+0000 7fafef200640 1 Processor -- start 2024-04-15T15:25:54.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.696+0000 7fafef200640 1 -- start start 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafef200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faff015b860 0x7faff006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafef200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faff006f320 0x7faff00b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafef200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff00cdfb0 con 0x7faff015b480 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafef200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7faff015f060 con 0x7faff015b860 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafef200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7faff015fec0 con 0x7faff006f320 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafe7e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faff015b860 0x7faff006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafe7e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faff015b860 0x7faff006ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49748/0 (socket says 172.21.15.150:49748) 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafe7e00640 1 -- 172.21.15.150:0/1277120213 learned_addr learned my addr 172.21.15.150:0/1277120213 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:54.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafee800640 1 --2- 172.21.15.150:0/1277120213 >> v2:172.21.15.187:3300/0 conn(0x7faff006f320 0x7faff00b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafe7e00640 1 -- 172.21.15.150:0/1277120213 >> v2:172.21.15.187:3300/0 conn(0x7faff006f320 msgr2=0x7faff00b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:54.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafe7e00640 1 --2- 172.21.15.150:0/1277120213 >> v2:172.21.15.187:3300/0 conn(0x7faff006f320 0x7faff00b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafe7e00640 1 -- 172.21.15.150:0/1277120213 >> v1:172.21.15.187:6789/0 conn(0x7faff015b480 legacy=0x7faff00cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:54.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.698+0000 7fafe7e00640 1 -- 172.21.15.150:0/1277120213 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff015f2b0 con 0x7faff015b860 2024-04-15T15:25:54.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.699+0000 7fafe7e00640 1 --2- 172.21.15.150:0/1277120213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b860 0x7faff006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7faff006ffc0 tx=0x7fafd802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e39a7143d2c3745 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:54.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.700+0000 7fafed400640 1 -- 172.21.15.150:0/1277120213 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8035020 con 0x7faff015b860 2024-04-15T15:25:54.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.700+0000 7fafed400640 1 -- 172.21.15.150:0/1277120213 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd8031870 con 0x7faff015b860 2024-04-15T15:25:54.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.700+0000 7fafed400640 1 -- 172.21.15.150:0/1277120213 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8031c40 con 0x7faff015b860 2024-04-15T15:25:54.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.701+0000 7fafef200640 1 -- 172.21.15.150:0/1277120213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b860 msgr2=0x7faff006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.701+0000 7fafef200640 1 --2- 172.21.15.150:0/1277120213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b860 0x7faff006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7faff006ffc0 tx=0x7fafd802b910 comp rx=0 tx=0).stop 2024-04-15T15:25:54.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.701+0000 7fafef200640 1 -- 172.21.15.150:0/1277120213 shutdown_connections 2024-04-15T15:25:54.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.701+0000 7fafef200640 1 --2- 172.21.15.150:0/1277120213 >> v2:172.21.15.187:3300/0 conn(0x7faff006f320 0x7faff00b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.701+0000 7fafef200640 1 --2- 172.21.15.150:0/1277120213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b860 0x7faff006ede0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.701+0000 7fafef200640 1 -- 172.21.15.150:0/1277120213 >> 172.21.15.150:0/1277120213 conn(0x7faff006b930 msgr2=0x7faff006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:54.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.701+0000 7fafef200640 1 -- 172.21.15.150:0/1277120213 shutdown_connections 2024-04-15T15:25:54.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.701+0000 7fafef200640 1 -- 172.21.15.150:0/1277120213 wait complete. 2024-04-15T15:25:54.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.703+0000 7fafef200640 1 Processor -- start 2024-04-15T15:25:54.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.703+0000 7fafef200640 1 -- start start 2024-04-15T15:25:54.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafef200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff006f320 0x7faff00c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafef200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b480 0x7faff00c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafef200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff015b860 0x7faff01e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafef200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faff015fb50 con 0x7faff015b480 2024-04-15T15:25:54.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafef200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faff015f870 con 0x7faff015b860 2024-04-15T15:25:54.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafef200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faff00cd550 con 0x7faff006f320 2024-04-15T15:25:54.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafe7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b480 0x7faff00c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafe7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b480 0x7faff00c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49758/0 (socket says 172.21.15.150:49758) 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.704+0000 7fafe7e00640 1 -- 172.21.15.150:0/3204093994 learned_addr learned my addr 172.21.15.150:0/3204093994 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafee800640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff015b860 0x7faff01e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafe7e00640 1 -- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff006f320 msgr2=0x7faff00c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafede00640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff006f320 0x7faff00c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafe7e00640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff006f320 0x7faff00c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafe7e00640 1 -- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff015b860 msgr2=0x7faff01e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafe7e00640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff015b860 0x7faff01e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafe7e00640 1 -- 172.21.15.150:0/3204093994 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff01e7e20 con 0x7faff015b480 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafee800640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff015b860 0x7faff01e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafe7e00640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b480 0x7faff00c7dd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fafd8000c00 tx=0x7fafd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.705+0000 7fafe6000640 1 -- 172.21.15.150:0/3204093994 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8040070 con 0x7faff015b480 2024-04-15T15:25:54.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.706+0000 7fafe6000640 1 -- 172.21.15.150:0/3204093994 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd803a420 con 0x7faff015b480 2024-04-15T15:25:54.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.706+0000 7fafe6000640 1 -- 172.21.15.150:0/3204093994 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8035040 con 0x7faff015b480 2024-04-15T15:25:54.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.706+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff01e8100 con 0x7faff015b480 2024-04-15T15:25:54.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.706+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01e86c0 con 0x7faff015b480 2024-04-15T15:25:54.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.707+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff00755c0 con 0x7faff015b480 2024-04-15T15:25:54.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.708+0000 7fafe6000640 1 -- 172.21.15.150:0/3204093994 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fafd8038020 con 0x7faff015b480 2024-04-15T15:25:54.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.709+0000 7fafe6000640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fafcc06c980 0x7fafcc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:54.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.709+0000 7fafe6000640 1 -- 172.21.15.150:0/3204093994 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fafd80adbb0 con 0x7faff015b480 2024-04-15T15:25:54.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.710+0000 7fafede00640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fafcc06c980 0x7fafcc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:54.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.711+0000 7fafede00640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fafcc06c980 0x7fafcc06ee40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fafe00015d0 tx=0x7fafe0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:54.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.714+0000 7fafe6000640 1 -- 172.21.15.150:0/3204093994 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafd80b4080 con 0x7faff015b480 2024-04-15T15:25:54.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.916+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faff006c0a0 con 0x7faff015b480 2024-04-15T15:25:54.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.919+0000 7fafe6000640 1 -- 172.21.15.150:0/3204093994 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16848 (secure 0 0 0) 0x7fafd807bfa0 con 0x7faff015b480 2024-04-15T15:25:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:25:54.921 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":36,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:33.990388+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-15T15:25:04.190900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-15T15:25:05.324314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/3327940880":"2024-04-16T15:25:33.990360+0000","172.21.15.150:6833/3327940880":"2024-04-16T15:25:33.990360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:25:54.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.922+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fafcc06c980 msgr2=0x7fafcc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.922+0000 7fafef200640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fafcc06c980 0x7fafcc06ee40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fafe00015d0 tx=0x7fafe0009040 comp rx=0 tx=0).stop 2024-04-15T15:25:54.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.922+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b480 msgr2=0x7faff00c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:54.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.922+0000 7fafef200640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b480 0x7faff00c7dd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fafd8000c00 tx=0x7fafd80046c0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.923+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 shutdown_connections 2024-04-15T15:25:54.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.923+0000 7fafef200640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff015b860 0x7faff01e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.923+0000 7fafef200640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff015b480 0x7faff00c7dd0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.923+0000 7fafef200640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fafcc06c980 0x7fafcc06ee40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.923+0000 7fafef200640 1 --2- 172.21.15.150:0/3204093994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faff006f320 0x7faff00c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:54.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.923+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 >> 172.21.15.150:0/3204093994 conn(0x7faff006b930 msgr2=0x7faff00b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:54.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.923+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 shutdown_connections 2024-04-15T15:25:54.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:54.923+0000 7fafef200640 1 -- 172.21.15.150:0/3204093994 wait complete. 2024-04-15T15:25:54.946 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-15T15:25:55.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.154+0000 7f15bd200640 1 Processor -- start 2024-04-15T15:25:55.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.155+0000 7f15bd200640 1 -- start start 2024-04-15T15:25:55.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.156+0000 7f15bd200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f15b809baf0 0x7f15b8070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.156+0000 7f15bd200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f15b815ee60 0x7f15b8071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.156+0000 7f15bd200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b80ccda0 con 0x7f15b815ea80 2024-04-15T15:25:55.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.156+0000 7f15bd200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f15b80cd6b0 con 0x7f15b815ee60 2024-04-15T15:25:55.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.156+0000 7f15bd200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f15b809bff0 con 0x7f15b809baf0 2024-04-15T15:25:55.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.157+0000 7f15b6800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f15b815ee60 0x7f15b8071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.157+0000 7f15b6800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f15b815ee60 0x7f15b8071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49766/0 (socket says 172.21.15.150:49766) 2024-04-15T15:25:55.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.157+0000 7f15b6800640 1 -- 172.21.15.150:0/2017273415 learned_addr learned my addr 172.21.15.150:0/2017273415 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:55.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.157+0000 7f15ad400640 1 --2- 172.21.15.150:0/2017273415 >> v2:172.21.15.187:3300/0 conn(0x7f15b809baf0 0x7f15b8070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.157+0000 7f15b6800640 1 -- 172.21.15.150:0/2017273415 >> v2:172.21.15.187:3300/0 conn(0x7f15b809baf0 msgr2=0x7f15b8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:55.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.157+0000 7f15b6800640 1 --2- 172.21.15.150:0/2017273415 >> v2:172.21.15.187:3300/0 conn(0x7f15b809baf0 0x7f15b8070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.157+0000 7f15b6800640 1 -- 172.21.15.150:0/2017273415 >> v1:172.21.15.187:6789/0 conn(0x7f15b815ea80 legacy=0x7f15b8070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:55.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.157+0000 7f15b6800640 1 -- 172.21.15.150:0/2017273415 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b80cd060 con 0x7f15b815ee60 2024-04-15T15:25:55.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.158+0000 7f15b6800640 1 --2- 172.21.15.150:0/2017273415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ee60 0x7f15b8071360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f15b80cd830 tx=0x7f15b002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5776f560a7f4f15b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:55.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.158+0000 7f15b5400640 1 -- 172.21.15.150:0/2017273415 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b002cb00 con 0x7f15b815ee60 2024-04-15T15:25:55.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.159+0000 7f15b5400640 1 -- 172.21.15.150:0/2017273415 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b002cca0 con 0x7f15b815ee60 2024-04-15T15:25:55.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.159+0000 7f15b5400640 1 -- 172.21.15.150:0/2017273415 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0034610 con 0x7f15b815ee60 2024-04-15T15:25:55.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.159+0000 7f15bd200640 1 -- 172.21.15.150:0/2017273415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ee60 msgr2=0x7f15b8071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:55.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.159+0000 7f15bd200640 1 --2- 172.21.15.150:0/2017273415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ee60 0x7f15b8071360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f15b80cd830 tx=0x7f15b002bf60 comp rx=0 tx=0).stop 2024-04-15T15:25:55.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.159+0000 7f15bd200640 1 -- 172.21.15.150:0/2017273415 shutdown_connections 2024-04-15T15:25:55.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.159+0000 7f15bd200640 1 --2- 172.21.15.150:0/2017273415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ee60 0x7f15b8071360 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.159+0000 7f15bd200640 1 --2- 172.21.15.150:0/2017273415 >> v2:172.21.15.187:3300/0 conn(0x7f15b809baf0 0x7f15b8070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.159+0000 7f15bd200640 1 -- 172.21.15.150:0/2017273415 >> 172.21.15.150:0/2017273415 conn(0x7f15b80d2750 msgr2=0x7f15b80d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:55.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.160+0000 7f15bd200640 1 -- 172.21.15.150:0/2017273415 shutdown_connections 2024-04-15T15:25:55.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.160+0000 7f15bd200640 1 -- 172.21.15.150:0/2017273415 wait complete. 2024-04-15T15:25:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.161+0000 7f15bd200640 1 Processor -- start 2024-04-15T15:25:55.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.161+0000 7f15bd200640 1 -- start start 2024-04-15T15:25:55.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.162+0000 7f15bd200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15b809baf0 0x7f15b80ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.162+0000 7f15bd200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ea80 0x7f15b815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.162+0000 7f15bd200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15b815ee60 0x7f15b80caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.162+0000 7f15bd200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15b806dca0 con 0x7f15b815ea80 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.162+0000 7f15bd200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15b806ca40 con 0x7f15b815ee60 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.162+0000 7f15bd200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f15b806de20 con 0x7f15b809baf0 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ea80 0x7f15b815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ea80 0x7f15b815ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49780/0 (socket says 172.21.15.150:49780) 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 -- 172.21.15.150:0/3022181058 learned_addr learned my addr 172.21.15.150:0/3022181058 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15b6800640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15b815ee60 0x7f15b80caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 -- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15b809baf0 msgr2=0x7f15b80ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15b809baf0 0x7f15b80ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 -- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15b815ee60 msgr2=0x7f15b80caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15b815ee60 0x7f15b80caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 -- 172.21.15.150:0/3022181058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b80cb970 con 0x7f15b815ea80 2024-04-15T15:25:55.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.163+0000 7f15ad400640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ea80 0x7f15b815ce90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15a000d830 tx=0x7f15a000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:55.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.164+0000 7f15aea00640 1 -- 172.21.15.150:0/3022181058 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a000b6c0 con 0x7f15b815ea80 2024-04-15T15:25:55.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.164+0000 7f15aea00640 1 -- 172.21.15.150:0/3022181058 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a0019070 con 0x7f15b815ea80 2024-04-15T15:25:55.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.164+0000 7f15aea00640 1 -- 172.21.15.150:0/3022181058 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a000ba80 con 0x7f15b815ea80 2024-04-15T15:25:55.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.164+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b80c9a70 con 0x7f15b815ea80 2024-04-15T15:25:55.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.164+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b815bf20 con 0x7f15b815ea80 2024-04-15T15:25:55.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.165+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b8068c80 con 0x7f15b815ea80 2024-04-15T15:25:55.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.166+0000 7f15aea00640 1 -- 172.21.15.150:0/3022181058 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f15a001e460 con 0x7f15b815ea80 2024-04-15T15:25:55.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.167+0000 7f15aea00640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f159406c980 0x7f159406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.167+0000 7f15aea00640 1 -- 172.21.15.150:0/3022181058 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f15a008c230 con 0x7f15b815ea80 2024-04-15T15:25:55.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.168+0000 7f15b5e00640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f159406c980 0x7f159406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.169+0000 7f15b5e00640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f159406c980 0x7f159406ee40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f15a8003cc0 tx=0x7f15a80079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:55.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.173+0000 7f15aea00640 1 -- 172.21.15.150:0/3022181058 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15a005a6a0 con 0x7f15b815ea80 2024-04-15T15:25:55.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.415+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f15b80b1760 con 0x7f15b815ea80 2024-04-15T15:25:55.418 INFO:tasks.ceph.mon.a.smithi150.stderr:2024-04-15T15:25:55.417+0000 7f8c07200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-15T15:25:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.420+0000 7f15aea00640 1 -- 172.21.15.150:0/3022181058 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7f15a0059fa0 con 0x7f15b815ea80 2024-04-15T15:25:55.422 INFO:teuthology.orchestra.run.smithi150.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f159406c980 msgr2=0x7f159406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f159406c980 0x7f159406ee40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f15a8003cc0 tx=0x7f15a80079b0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ea80 msgr2=0x7f15b815ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ea80 0x7f15b815ce90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15a000d830 tx=0x7f15a000dcf0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 shutdown_connections 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15b815ee60 0x7f15b80caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b815ea80 0x7f15b815ce90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f159406c980 0x7f159406ee40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 --2- 172.21.15.150:0/3022181058 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15b809baf0 0x7f15b80ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 >> 172.21.15.150:0/3022181058 conn(0x7f15b80d2750 msgr2=0x7f15b80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 shutdown_connections 2024-04-15T15:25:55.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.423+0000 7f15bd200640 1 -- 172.21.15.150:0/3022181058 wait complete. 2024-04-15T15:25:55.449 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-15T15:25:55.450 INFO:tasks.ceph.mds.a.smithi150.stderr: -2727> 2024-04-15T15:25:39.895+0000 7f03e2200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-15T15:25:55.509 INFO:tasks.ceph.mds.a.smithi150.stdout:starting mds.a at 2024-04-15T15:25:55.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.637+0000 7f018c000640 1 Processor -- start 2024-04-15T15:25:55.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.637+0000 7f018c000640 1 -- start start 2024-04-15T15:25:55.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.638+0000 7f018c000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0184145170 0x7f01840cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.638+0000 7f018c000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f01840ce070 0x7f01840b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.638+0000 7f018c000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840b59f0 0x7f018406f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.638+0000 7f018c000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f0184145550 con 0x7f0184145170 2024-04-15T15:25:55.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.638+0000 7f018c000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f01840b4410 con 0x7f01840ce070 2024-04-15T15:25:55.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.638+0000 7f018c000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f01840b46f0 con 0x7f01840b59f0 2024-04-15T15:25:55.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0189600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0184145170 0x7f01840cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0189600640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0184145170 0x7f01840cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49808/0 (socket says 172.21.15.150:49808) 2024-04-15T15:25:55.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0189600640 1 -- 172.21.15.150:0/4227036691 learned_addr learned my addr 172.21.15.150:0/4227036691 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:55.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f018a000640 1 --2- 172.21.15.150:0/4227036691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840b59f0 0x7f018406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0188c00640 1 --2- 172.21.15.150:0/4227036691 >> v2:172.21.15.187:3300/0 conn(0x7f01840ce070 0x7f01840b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0189600640 1 -- 172.21.15.150:0/4227036691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840b59f0 msgr2=0x7f018406f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:55.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0189600640 1 --2- 172.21.15.150:0/4227036691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840b59f0 0x7f018406f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0189600640 1 -- 172.21.15.150:0/4227036691 >> v2:172.21.15.187:3300/0 conn(0x7f01840ce070 msgr2=0x7f01840b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:55.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0189600640 1 --2- 172.21.15.150:0/4227036691 >> v2:172.21.15.187:3300/0 conn(0x7f01840ce070 0x7f01840b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.639+0000 7f0189600640 1 -- 172.21.15.150:0/4227036691 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01840b49d0 con 0x7f0184145170 2024-04-15T15:25:55.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.640+0000 7f0189600640 1 --2- 172.21.15.150:0/4227036691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840cdb30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f01840b4900 tx=0x7f017402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ab6c425bf66c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:55.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f0183e00640 1 -- 172.21.15.150:0/4227036691 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174035020 con 0x7f0184145170 2024-04-15T15:25:55.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f0183e00640 1 -- 172.21.15.150:0/4227036691 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0174031640 con 0x7f0184145170 2024-04-15T15:25:55.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f0183e00640 1 -- 172.21.15.150:0/4227036691 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017402d040 con 0x7f0184145170 2024-04-15T15:25:55.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f018c000640 1 -- 172.21.15.150:0/4227036691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 msgr2=0x7f01840cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:55.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f018c000640 1 --2- 172.21.15.150:0/4227036691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840cdb30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f01840b4900 tx=0x7f017402b8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f018c000640 1 -- 172.21.15.150:0/4227036691 shutdown_connections 2024-04-15T15:25:55.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f018c000640 1 --2- 172.21.15.150:0/4227036691 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840b59f0 0x7f018406f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f018c000640 1 --2- 172.21.15.150:0/4227036691 >> v2:172.21.15.187:3300/0 conn(0x7f01840ce070 0x7f01840b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.641+0000 7f018c000640 1 --2- 172.21.15.150:0/4227036691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840cdb30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.642+0000 7f018c000640 1 -- 172.21.15.150:0/4227036691 >> 172.21.15.150:0/4227036691 conn(0x7f01840b2460 msgr2=0x7f01840b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:55.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.642+0000 7f018c000640 1 -- 172.21.15.150:0/4227036691 shutdown_connections 2024-04-15T15:25:55.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.642+0000 7f018c000640 1 -- 172.21.15.150:0/4227036691 wait complete. 2024-04-15T15:25:55.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.644+0000 7f018c000640 1 Processor -- start 2024-04-15T15:25:55.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.644+0000 7f018c000640 1 -- start start 2024-04-15T15:25:55.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018c000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01840b59f0 0x7f01841e6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018c000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840ce070 0x7f01840ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018c000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018c000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f01840b4030 con 0x7f0184145170 2024-04-15T15:25:55.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018c000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01840b4f90 con 0x7f01840b59f0 2024-04-15T15:25:55.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018c000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f01840b41b0 con 0x7f01840ce070 2024-04-15T15:25:55.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49818/0 (socket says 172.21.15.150:49818) 2024-04-15T15:25:55.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f018a000640 1 -- 172.21.15.150:0/3315811089 learned_addr learned my addr 172.21.15.150:0/3315811089 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:55.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.645+0000 7f0189600640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01840b59f0 0x7f01841e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f0188c00640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840ce070 0x7f01840ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f018a000640 1 -- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840ce070 msgr2=0x7f01840ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:55.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f018a000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840ce070 0x7f01840ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f018a000640 1 -- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01840b59f0 msgr2=0x7f01841e6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:55.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f018a000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01840b59f0 0x7f01841e6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:55.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f018a000640 1 -- 172.21.15.150:0/3315811089 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01841e3ca0 con 0x7f0184145170 2024-04-15T15:25:55.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f0188c00640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840ce070 0x7f01840ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:55.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f018a000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840adc00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f017c00cd70 tx=0x7f017c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:55.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f0181600640 1 -- 172.21.15.150:0/3315811089 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c003b90 con 0x7f0184145170 2024-04-15T15:25:55.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f0181600640 1 -- 172.21.15.150:0/3315811089 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017c003d30 con 0x7f0184145170 2024-04-15T15:25:55.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f0181600640 1 -- 172.21.15.150:0/3315811089 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c004240 con 0x7f0184145170 2024-04-15T15:25:55.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01841e2490 con 0x7f0184145170 2024-04-15T15:25:55.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.646+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018412feb0 con 0x7f0184145170 2024-04-15T15:25:55.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.648+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0184068c80 con 0x7f0184145170 2024-04-15T15:25:55.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.649+0000 7f0181600640 1 -- 172.21.15.150:0/3315811089 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f017c0043e0 con 0x7f0184145170 2024-04-15T15:25:55.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.649+0000 7f0181600640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f016406c9b0 0x7f016406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:55.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.649+0000 7f0181600640 1 -- 172.21.15.150:0/3315811089 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5881+0+0 (secure 0 0 0) 0x7f017c08c860 con 0x7f0184145170 2024-04-15T15:25:55.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.650+0000 7f0189600640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f016406c9b0 0x7f016406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.651+0000 7f0189600640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f016406c9b0 0x7f016406ee70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0174036000 tx=0x7f017403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:55.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.655+0000 7f0181600640 1 -- 172.21.15.150:0/3315811089 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01841e3ca0 con 0x7f0184145170 2024-04-15T15:25:55.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:55.881+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f01840b2a90 con 0x7f0184145170 2024-04-15T15:25:56.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.424+0000 7f0181600640 1 -- 172.21.15.150:0/3315811089 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f017c05aa70 con 0x7f0184145170 2024-04-15T15:25:56.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f016406c9b0 msgr2=0x7f016406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:56.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f016406c9b0 0x7f016406ee70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0174036000 tx=0x7f017403c040 comp rx=0 tx=0).stop 2024-04-15T15:25:56.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 msgr2=0x7f01840adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:56.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840adc00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f017c00cd70 tx=0x7f017c00a660 comp rx=0 tx=0).stop 2024-04-15T15:25:56.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 shutdown_connections 2024-04-15T15:25:56.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0184145170 0x7f01840adc00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f016406c9b0 0x7f016406ee70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01840ce070 0x7f01840ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 --2- 172.21.15.150:0/3315811089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01840b59f0 0x7f01841e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.428+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 >> 172.21.15.150:0/3315811089 conn(0x7f01840b2460 msgr2=0x7f018406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:56.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.429+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 shutdown_connections 2024-04-15T15:25:56.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.429+0000 7f018c000640 1 -- 172.21.15.150:0/3315811089 wait complete. 2024-04-15T15:25:56.452 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-15T15:25:56.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.664+0000 7f8d0ae00640 1 Processor -- start 2024-04-15T15:25:56.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.665+0000 7f8d0ae00640 1 -- start start 2024-04-15T15:25:56.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.666+0000 7f8d0ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d04146240 0x7f8d040d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:56.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.666+0000 7f8d0ae00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d040b6bc0 0x7f8d040b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:56.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.666+0000 7f8d0ae00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8d0406b930 0x7f8d0406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:56.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.666+0000 7f8d0ae00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f8d040b7560 con 0x7f8d0406b930 2024-04-15T15:25:56.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.666+0000 7f8d0ae00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8d04146620 con 0x7f8d040b6bc0 2024-04-15T15:25:56.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.666+0000 7f8d0ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d040b02f0 con 0x7f8d04146240 2024-04-15T15:25:56.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d08e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8d0406b930 0x7f8d0406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:56.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d08e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8d0406b930 0x7f8d0406bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49834/0 (socket says 172.21.15.150:49834) 2024-04-15T15:25:56.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d08e00640 1 -- 172.21.15.150:0/1646436569 learned_addr learned my addr 172.21.15.150:0/1646436569 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:56.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d03e00640 1 --2- 172.21.15.150:0/1646436569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d04146240 0x7f8d040d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:56.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8cfbe00640 1 --2- 172.21.15.150:0/1646436569 >> v2:172.21.15.187:3300/0 conn(0x7f8d040b6bc0 0x7f8d040b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:56.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d08e00640 1 -- 172.21.15.150:0/1646436569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d04146240 msgr2=0x7f8d040d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:56.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d08e00640 1 --2- 172.21.15.150:0/1646436569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d04146240 0x7f8d040d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d08e00640 1 -- 172.21.15.150:0/1646436569 >> v2:172.21.15.187:3300/0 conn(0x7f8d040b6bc0 msgr2=0x7f8d040b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:56.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d08e00640 1 --2- 172.21.15.150:0/1646436569 >> v2:172.21.15.187:3300/0 conn(0x7f8d040b6bc0 0x7f8d040b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.667+0000 7f8d08e00640 1 -- 172.21.15.150:0/1646436569 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d040b05d0 con 0x7f8d0406b930 2024-04-15T15:25:56.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.669+0000 7f8d08e00640 1 --2- 172.21.15.150:0/1646436569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0406b930 0x7f8d0406bd60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8d0406f170 tx=0x7f8cfc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dbb3e47c913ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:56.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.669+0000 7f8d03400640 1 -- 172.21.15.150:0/1646436569 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc02ca70 con 0x7f8d0406b930 2024-04-15T15:25:56.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.669+0000 7f8d03400640 1 -- 172.21.15.150:0/1646436569 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc02cc10 con 0x7f8d0406b930 2024-04-15T15:25:56.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.669+0000 7f8d03400640 1 -- 172.21.15.150:0/1646436569 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0343f0 con 0x7f8d0406b930 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.669+0000 7f8d0ae00640 1 -- 172.21.15.150:0/1646436569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0406b930 msgr2=0x7f8d0406bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.670+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/1646436569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0406b930 0x7f8d0406bd60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8d0406f170 tx=0x7f8cfc02bed0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.670+0000 7f8d0ae00640 1 -- 172.21.15.150:0/1646436569 shutdown_connections 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.670+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/1646436569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0406b930 0x7f8d0406bd60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.670+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/1646436569 >> v2:172.21.15.187:3300/0 conn(0x7f8d040b6bc0 0x7f8d040b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.670+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/1646436569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d04146240 0x7f8d040d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.670+0000 7f8d0ae00640 1 -- 172.21.15.150:0/1646436569 >> 172.21.15.150:0/1646436569 conn(0x7f8d0409aa40 msgr2=0x7f8d0409ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.670+0000 7f8d0ae00640 1 -- 172.21.15.150:0/1646436569 shutdown_connections 2024-04-15T15:25:56.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.670+0000 7f8d0ae00640 1 -- 172.21.15.150:0/1646436569 wait complete. 2024-04-15T15:25:56.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.671+0000 7f8d0ae00640 1 Processor -- start 2024-04-15T15:25:56.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.672+0000 7f8d0ae00640 1 -- start start 2024-04-15T15:25:56.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.672+0000 7f8d0ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d0406b930 0x7f8d04155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d0ae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d040b6bc0 0x7f8d0415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d0ae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d04146240 0x7f8d04153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d0ae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8d040b0e70 con 0x7f8d04146240 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d0ae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d040b0b90 con 0x7f8d040b6bc0 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d0ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d040affc0 con 0x7f8d0406b930 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d08e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d04146240 0x7f8d04153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d08e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d04146240 0x7f8d04153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49840/0 (socket says 172.21.15.150:49840) 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d08e00640 1 -- 172.21.15.150:0/4136298519 learned_addr learned my addr 172.21.15.150:0/4136298519 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8cfbe00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d040b6bc0 0x7f8d0415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d03e00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d0406b930 0x7f8d04155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d08e00640 1 -- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d0406b930 msgr2=0x7f8d04155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d08e00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d0406b930 0x7f8d04155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d08e00640 1 -- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d040b6bc0 msgr2=0x7f8d0415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:56.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d08e00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d040b6bc0 0x7f8d0415f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:56.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.673+0000 7f8d08e00640 1 -- 172.21.15.150:0/4136298519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0415e8c0 con 0x7f8d04146240 2024-04-15T15:25:56.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.674+0000 7f8d08e00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d04146240 0x7f8d04153170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009f40 tx=0x7f8cfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:56.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.674+0000 7f8d00c00640 1 -- 172.21.15.150:0/4136298519 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc043020 con 0x7f8d04146240 2024-04-15T15:25:56.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.674+0000 7f8d00c00640 1 -- 172.21.15.150:0/4136298519 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc03e450 con 0x7f8d04146240 2024-04-15T15:25:56.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.674+0000 7f8d00c00640 1 -- 172.21.15.150:0/4136298519 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc03e840 con 0x7f8d04146240 2024-04-15T15:25:56.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.674+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0413e280 con 0x7f8d04146240 2024-04-15T15:25:56.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.674+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d040b44e0 con 0x7f8d04146240 2024-04-15T15:25:56.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.675+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d04068db0 con 0x7f8d04146240 2024-04-15T15:25:56.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.676+0000 7f8d00c00640 1 -- 172.21.15.150:0/4136298519 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8cfc046020 con 0x7f8d04146240 2024-04-15T15:25:56.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.677+0000 7f8d00c00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ce006c980 0x7f8ce006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:56.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.677+0000 7f8d00c00640 1 -- 172.21.15.150:0/4136298519 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5881+0+0 (secure 0 0 0) 0x7f8cfc0ae4c0 con 0x7f8d04146240 2024-04-15T15:25:56.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.678+0000 7f8d03e00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ce006c980 0x7f8ce006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:56.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.679+0000 7f8d03e00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ce006c980 0x7f8ce006ee40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4006fd0 tx=0x7f8cf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:56.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.681+0000 7f8d00c00640 1 -- 172.21.15.150:0/4136298519 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cfc04d030 con 0x7f8d04146240 2024-04-15T15:25:56.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:56.886+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d0415fd60 con 0x7f8d04146240 2024-04-15T15:25:57.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.427+0000 7f8d00c00640 1 -- 172.21.15.150:0/4136298519 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) ==== 181+0+0 (secure 0 0 0) 0x7f8cfc07c7e0 con 0x7f8d04146240 2024-04-15T15:25:57.429 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_metadata' removed 2024-04-15T15:25:57.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ce006c980 msgr2=0x7f8ce006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:57.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ce006c980 0x7f8ce006ee40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4006fd0 tx=0x7f8cf4008040 comp rx=0 tx=0).stop 2024-04-15T15:25:57.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d04146240 msgr2=0x7f8d04153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:57.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d04146240 0x7f8d04153170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009f40 tx=0x7f8cfc004a40 comp rx=0 tx=0).stop 2024-04-15T15:25:57.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 shutdown_connections 2024-04-15T15:25:57.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d04146240 0x7f8d04153170 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d040b6bc0 0x7f8d0415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8ce006c980 0x7f8ce006ee40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 --2- 172.21.15.150:0/4136298519 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d0406b930 0x7f8d04155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.432+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 >> 172.21.15.150:0/4136298519 conn(0x7f8d0409aa40 msgr2=0x7f8d041443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:57.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.433+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 shutdown_connections 2024-04-15T15:25:57.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.433+0000 7f8d0ae00640 1 -- 172.21.15.150:0/4136298519 wait complete. 2024-04-15T15:25:57.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-15T15:25:57.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.667+0000 7fc674a00640 1 Processor -- start 2024-04-15T15:25:57.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.668+0000 7fc674a00640 1 -- start start 2024-04-15T15:25:57.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.669+0000 7fc674a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc6700cd850 0x7fc6701673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:57.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.669+0000 7fc674a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6700ce000 con 0x7fc6701462f0 2024-04-15T15:25:57.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.669+0000 7fc674a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6700cdd20 con 0x7fc670072860 2024-04-15T15:25:57.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.669+0000 7fc674a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc6700a9640 con 0x7fc6700cd850 2024-04-15T15:25:57.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66de00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc6701462f0 0x7fc670072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38340/0 (socket says 172.21.15.150:38340) 2024-04-15T15:25:57.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66de00640 1 -- 172.21.15.150:0/1140582978 learned_addr learned my addr 172.21.15.150:0/1140582978 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:57.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66e800640 1 --2- 172.21.15.150:0/1140582978 >> v2:172.21.15.187:3300/0 conn(0x7fc6700cd850 0x7fc6701673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:57.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164162167 0 0) 0x7fc6700ce000 con 0x7fc6701462f0 2024-04-15T15:25:57.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc654003520 con 0x7fc6701462f0 2024-04-15T15:25:57.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4071011207 0 0) 0x7fc654003520 con 0x7fc6701462f0 2024-04-15T15:25:57.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 >> v2:172.21.15.187:3300/0 conn(0x7fc6700cd850 msgr2=0x7fc6701673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66d400640 1 --2- 172.21.15.150:0/1140582978 >> v2:172.21.15.187:3300/0 conn(0x7fc6700cd850 0x7fc6701673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 >> v1:172.21.15.187:6789/0 conn(0x7fc670072860 legacy=0x7fc6700cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67015cf10 con 0x7fc6701462f0 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.670+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fc6600019f0 con 0x7fc6701462f0 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 >> v1:172.21.15.150:6789/0 conn(0x7fc6701462f0 legacy=0x7fc670072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66d400640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6540039b0 0x7fc654003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66d400640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc654004800 0x7fc654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66d400640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6540051c0 0x7fc65400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc65400c040 con 0x7fc6540051c0 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc65400c1f0 con 0x7fc654004800 2024-04-15T15:25:57.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc65400c3f0 con 0x7fc6540039b0 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66e800640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6540051c0 0x7fc65400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc665400640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc654004800 0x7fc654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66de00640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6540039b0 0x7fc654003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66e800640 1 -- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6540039b0 msgr2=0x7fc654003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66e800640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6540039b0 0x7fc654003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66e800640 1 -- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc654004800 msgr2=0x7fc654004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66e800640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc654004800 0x7fc654004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc66e800640 1 -- 172.21.15.150:0/1140582978 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc654003520 con 0x7fc6540051c0 2024-04-15T15:25:57.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.671+0000 7fc665400640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc654004800 0x7fc654004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.672+0000 7fc66e800640 1 -- 172.21.15.150:0/1140582978 reap_dead start 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.672+0000 7fc66e800640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6540051c0 0x7fc65400b9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc6700730c0 tx=0x7fc654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d09f6d43e69c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668039070 con 0x7fc6540051c0 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66802c680 con 0x7fc6540051c0 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc66d400640 1 -- 172.21.15.150:0/1140582978 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668009690 con 0x7fc6540051c0 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc674a00640 1 -- 172.21.15.150:0/1140582978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6540051c0 msgr2=0x7fc65400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc674a00640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6540051c0 0x7fc65400b9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc6700730c0 tx=0x7fc654005830 comp rx=0 tx=0).stop 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc674a00640 1 -- 172.21.15.150:0/1140582978 shutdown_connections 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc674a00640 1 --2- 172.21.15.150:0/1140582978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6540051c0 0x7fc65400b9c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc674a00640 1 -- 172.21.15.150:0/1140582978 >> 172.21.15.150:0/1140582978 conn(0x7fc6700b2460 msgr2=0x7fc6700b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.673+0000 7fc674a00640 1 -- 172.21.15.150:0/1140582978 shutdown_connections 2024-04-15T15:25:57.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.674+0000 7fc674a00640 1 -- 172.21.15.150:0/1140582978 wait complete. 2024-04-15T15:25:57.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.675+0000 7fc674a00640 1 Processor -- start 2024-04-15T15:25:57.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.675+0000 7fc674a00640 1 -- start start 2024-04-15T15:25:57.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.676+0000 7fc674a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6540051c0 0x7fc6701587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:57.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.676+0000 7fc674a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc67013c060 0x7fc67013d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.676+0000 7fc674a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc67013a870 0x7fc67013f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.676+0000 7fc674a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6700cd140 con 0x7fc67013a870 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.676+0000 7fc674a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc67015d4d0 con 0x7fc6540051c0 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.676+0000 7fc674a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6700cd2c0 con 0x7fc67013c060 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc67013a870 0x7fc67013f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc67013a870 0x7fc67013f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50948/0 (socket says 172.21.15.150:50948) 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 -- 172.21.15.150:0/2679588574 learned_addr learned my addr 172.21.15.150:0/2679588574 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 -- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc67013c060 msgr2=0x7fc67013d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66de00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6540051c0 0x7fc6701587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc67013c060 0x7fc67013d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 -- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6540051c0 msgr2=0x7fc6701587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6540051c0 0x7fc6701587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:57.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 -- 172.21.15.150:0/2679588574 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67013ad20 con 0x7fc67013a870 2024-04-15T15:25:57.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.677+0000 7fc66e800640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc67013a870 0x7fc67013f7f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc66802c2f0 tx=0x7fc668004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:57.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.678+0000 7fc666a00640 1 -- 172.21.15.150:0/2679588574 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668045070 con 0x7fc67013a870 2024-04-15T15:25:57.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.678+0000 7fc666a00640 1 -- 172.21.15.150:0/2679588574 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66803e890 con 0x7fc67013a870 2024-04-15T15:25:57.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.678+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67013afc0 con 0x7fc67013a870 2024-04-15T15:25:57.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.678+0000 7fc666a00640 1 -- 172.21.15.150:0/2679588574 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668039040 con 0x7fc67013a870 2024-04-15T15:25:57.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.678+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67013b580 con 0x7fc67013a870 2024-04-15T15:25:57.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.679+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc670068db0 con 0x7fc67013a870 2024-04-15T15:25:57.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.680+0000 7fc666a00640 1 -- 172.21.15.150:0/2679588574 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc668048070 con 0x7fc67013a870 2024-04-15T15:25:57.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.681+0000 7fc666a00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc64c06c950 0x7fc64c06ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:57.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.681+0000 7fc666a00640 1 -- 172.21.15.150:0/2679588574 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5443+0+0 (secure 0 0 0) 0x7fc6680ac930 con 0x7fc67013a870 2024-04-15T15:25:57.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.681+0000 7fc66de00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc64c06c950 0x7fc64c06ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:57.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.683+0000 7fc66de00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc64c06c950 0x7fc64c06ee10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc660001350 tx=0x7fc660009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:57.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.685+0000 7fc666a00640 1 -- 172.21.15.150:0/2679588574 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc66807a720 con 0x7fc67013a870 2024-04-15T15:25:57.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:57.886+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc67006fcc0 con 0x7fc67013a870 2024-04-15T15:25:58.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.441+0000 7fc666a00640 1 -- 172.21.15.150:0/2679588574 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) ==== 169+0+0 (secure 0 0 0) 0x7fc66807ea20 con 0x7fc67013a870 2024-04-15T15:25:58.443 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data' removed 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc64c06c950 msgr2=0x7fc64c06ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc64c06c950 0x7fc64c06ee10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc660001350 tx=0x7fc660009040 comp rx=0 tx=0).stop 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc67013a870 msgr2=0x7fc67013f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc67013a870 0x7fc67013f7f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc66802c2f0 tx=0x7fc668004730 comp rx=0 tx=0).stop 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 shutdown_connections 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc67013a870 0x7fc67013f7f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc64c06c950 0x7fc64c06ee10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc67013c060 0x7fc67013d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 --2- 172.21.15.150:0/2679588574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6540051c0 0x7fc6701587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.446+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 >> 172.21.15.150:0/2679588574 conn(0x7fc6700b2460 msgr2=0x7fc6700b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:58.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.447+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 shutdown_connections 2024-04-15T15:25:58.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.447+0000 7fc674a00640 1 -- 172.21.15.150:0/2679588574 wait complete. 2024-04-15T15:25:58.474 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_quiesce_splitauth 2024-04-15T15:25:58.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.679+0000 7f720d800640 1 Processor -- start 2024-04-15T15:25:58.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.679+0000 7f720d800640 1 -- start start 2024-04-15T15:25:58.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.681+0000 7f720d800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7208163520 0x7f720815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:58.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.681+0000 7f720d800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720815ffe0 con 0x7f7208145070 2024-04-15T15:25:58.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.681+0000 7f720d800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7208162a40 con 0x7f7208072860 2024-04-15T15:25:58.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.681+0000 7f720d800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f72080afde0 con 0x7f7208163520 2024-04-15T15:25:58.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.681+0000 7f7206a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7208145070 0x7f7208072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38346/0 (socket says 172.21.15.150:38346) 2024-04-15T15:25:58.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.681+0000 7f7206a00640 1 -- 172.21.15.150:0/2023283395 learned_addr learned my addr 172.21.15.150:0/2023283395 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.681+0000 7f7207400640 1 --2- 172.21.15.150:0/2023283395 >> v2:172.21.15.187:3300/0 conn(0x7f7208163520 0x7f720815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538588674 0 0) 0x7f720815ffe0 con 0x7f7208145070 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71ec003520 con 0x7f7208145070 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3250503930 0 0) 0x7f71ec003520 con 0x7f7208145070 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 >> v2:172.21.15.187:3300/0 conn(0x7f7208163520 msgr2=0x7f720815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 --2- 172.21.15.150:0/2023283395 >> v2:172.21.15.187:3300/0 conn(0x7f7208163520 0x7f720815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 >> v1:172.21.15.187:6789/0 conn(0x7f7208072860 legacy=0x7f720815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7208162d20 con 0x7f7208145070 2024-04-15T15:25:58.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f71f8001670 con 0x7f7208145070 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 >> v1:172.21.15.150:6789/0 conn(0x7f7208145070 legacy=0x7f7208072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f71ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec004800 0x7f71ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec0051c0 0x7f71ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f72080afde0 con 0x7f71ec0039b0 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec00c160 con 0x7f71ec004800 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec00c360 con 0x7f71ec0051c0 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.682+0000 7f7206a00640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f71ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.683+0000 7f7206000640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec004800 0x7f71ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.683+0000 7f7206a00640 1 -- 172.21.15.150:0/2023283395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec0051c0 msgr2=0x7f71ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.683+0000 7f7206a00640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec0051c0 0x7f71ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.683+0000 7f7206a00640 1 -- 172.21.15.150:0/2023283395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec004800 msgr2=0x7f71ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:58.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.683+0000 7f7206a00640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec004800 0x7f71ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.683+0000 7f7206a00640 1 -- 172.21.15.150:0/2023283395 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec003520 con 0x7f71ec0039b0 2024-04-15T15:25:58.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.683+0000 7f7206000640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec004800 0x7f71ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:58.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.683+0000 7f7207400640 1 -- 172.21.15.150:0/2023283395 reap_dead start 2024-04-15T15:25:58.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.684+0000 7f7206a00640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f71ec003dd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c240 tx=0x7f71ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6c0b326af8d32d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:58.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.684+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8007cf0 con 0x7f71ec0039b0 2024-04-15T15:25:58.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.684+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8030070 con 0x7f71ec0039b0 2024-04-15T15:25:58.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.684+0000 7f7205600640 1 -- 172.21.15.150:0/2023283395 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8005d70 con 0x7f71ec0039b0 2024-04-15T15:25:58.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.685+0000 7f720d800640 1 -- 172.21.15.150:0/2023283395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 msgr2=0x7f71ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.685+0000 7f720d800640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f71ec003dd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c240 tx=0x7f71ec005830 comp rx=0 tx=0).stop 2024-04-15T15:25:58.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.685+0000 7f720d800640 1 -- 172.21.15.150:0/2023283395 shutdown_connections 2024-04-15T15:25:58.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.685+0000 7f720d800640 1 --2- 172.21.15.150:0/2023283395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f71ec003dd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.685+0000 7f720d800640 1 -- 172.21.15.150:0/2023283395 >> 172.21.15.150:0/2023283395 conn(0x7f72080b2460 msgr2=0x7f72080b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:58.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.685+0000 7f720d800640 1 -- 172.21.15.150:0/2023283395 shutdown_connections 2024-04-15T15:25:58.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.685+0000 7f720d800640 1 -- 172.21.15.150:0/2023283395 wait complete. 2024-04-15T15:25:58.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.687+0000 7f720d800640 1 Processor -- start 2024-04-15T15:25:58.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.687+0000 7f720d800640 1 -- start start 2024-04-15T15:25:58.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f720d800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f7208142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:58.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f720d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7208140e10 0x7f720815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:58.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f720d800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f720813dbf0 0x7f720813e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f720d800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7208145450 con 0x7f71ec0039b0 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f720d800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72081627f0 con 0x7f720813dbf0 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f720d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7208072150 con 0x7f7208140e10 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f7206a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f7208142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f7206a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f7208142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50960/0 (socket says 172.21.15.150:50960) 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f7206a00640 1 -- 172.21.15.150:0/556578662 learned_addr learned my addr 172.21.15.150:0/556578662 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.688+0000 7f7207400640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f720813dbf0 0x7f720813e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f7206000640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7208140e10 0x7f720815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f7206a00640 1 -- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7208140e10 msgr2=0x7f720815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f7206a00640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7208140e10 0x7f720815c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f7206a00640 1 -- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f720813dbf0 msgr2=0x7f720813e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f7206a00640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f720813dbf0 0x7f720813e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f7206a00640 1 -- 172.21.15.150:0/556578662 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72081413d0 con 0x7f71ec0039b0 2024-04-15T15:25:58.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f7207400640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f720813dbf0 0x7f720813e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:58.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f7206a00640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f7208142470 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f71f802c500 tx=0x7f7208162ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:58.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f71fea00640 1 -- 172.21.15.150:0/556578662 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8045070 con 0x7f71ec0039b0 2024-04-15T15:25:58.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f71fea00640 1 -- 172.21.15.150:0/556578662 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8000ed0 con 0x7f71ec0039b0 2024-04-15T15:25:58.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f71fea00640 1 -- 172.21.15.150:0/556578662 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8001c30 con 0x7f71ec0039b0 2024-04-15T15:25:58.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72081416b0 con 0x7f71ec0039b0 2024-04-15T15:25:58.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.689+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7208141c70 con 0x7f71ec0039b0 2024-04-15T15:25:58.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.691+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72080b21c0 con 0x7f71ec0039b0 2024-04-15T15:25:58.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.692+0000 7f71fea00640 1 -- 172.21.15.150:0/556578662 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f71f8007070 con 0x7f71ec0039b0 2024-04-15T15:25:58.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.692+0000 7f71fea00640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71e806c9b0 0x7f71e806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:58.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.693+0000 7f71fea00640 1 -- 172.21.15.150:0/556578662 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f71f80acbf0 con 0x7f71ec0039b0 2024-04-15T15:25:58.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.693+0000 7f7206000640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71e806c9b0 0x7f71e806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:58.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.694+0000 7f7206000640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71e806c9b0 0x7f71e806ee70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71f4000c00 tx=0x7f71f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:58.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.697+0000 7f71fea00640 1 -- 172.21.15.150:0/556578662 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72080b21c0 con 0x7f71ec0039b0 2024-04-15T15:25:58.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.900+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_quiesce_splitauth"} v 0) -- 0x7f7208068c80 con 0x7f71ec0039b0 2024-04-15T15:25:58.909 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.908+0000 7f71fea00640 1 -- 172.21.15.150:0/556578662 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_quiesce_splitauth"}]=0 v13) ==== 110+0+0 (secure 0 0 0) 0x7f71f807b2d0 con 0x7f71ec0039b0 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71e806c9b0 msgr2=0x7f71e806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71e806c9b0 0x7f71e806ee70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71f4000c00 tx=0x7f71f4009040 comp rx=0 tx=0).stop 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 msgr2=0x7f7208142470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f7208142470 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f71f802c500 tx=0x7f7208162ef0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 shutdown_connections 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f720813dbf0 0x7f720813e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71e806c9b0 0x7f71e806ee70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7208140e10 0x7f720815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 --2- 172.21.15.150:0/556578662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ec0039b0 0x7f7208142470 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 >> 172.21.15.150:0/556578662 conn(0x7f72080b2460 msgr2=0x7f720806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 shutdown_connections 2024-04-15T15:25:58.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:58.911+0000 7f720d800640 1 -- 172.21.15.150:0/556578662 wait complete. 2024-04-15T15:25:58.933 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-15T15:25:59.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.130+0000 7f55b8c00640 1 Processor -- start 2024-04-15T15:25:59.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.130+0000 7f55b8c00640 1 -- start start 2024-04-15T15:25:59.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.132+0000 7f55b8c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f55b4072860 0x7f55b415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.132+0000 7f55b8c00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b415ffe0 con 0x7f55b4163520 2024-04-15T15:25:59.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.132+0000 7f55b8c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b4162a40 con 0x7f55b4145070 2024-04-15T15:25:59.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.132+0000 7f55b8c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f55b40afde0 con 0x7f55b4072860 2024-04-15T15:25:59.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.132+0000 7f55b2800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f55b4163520 0x7f55b415f6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38360/0 (socket says 172.21.15.150:38360) 2024-04-15T15:25:59.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.132+0000 7f55b2800640 1 -- 172.21.15.150:0/321254884 learned_addr learned my addr 172.21.15.150:0/321254884 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:25:59.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b1400640 1 --2- 172.21.15.150:0/321254884 >> v2:172.21.15.187:3300/0 conn(0x7f55b4072860 0x7f55b415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1868682415 0 0) 0x7f55b415ffe0 con 0x7f55b4163520 2024-04-15T15:25:59.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5598003520 con 0x7f55b4163520 2024-04-15T15:25:59.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2708814108 0 0) 0x7f5598003520 con 0x7f55b4163520 2024-04-15T15:25:59.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 >> v2:172.21.15.187:3300/0 conn(0x7f55b4072860 msgr2=0x7f55b415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 --2- 172.21.15.150:0/321254884 >> v2:172.21.15.187:3300/0 conn(0x7f55b4072860 0x7f55b415f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 >> v1:172.21.15.187:6789/0 conn(0x7f55b4145070 legacy=0x7f55b4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b4162d20 con 0x7f55b4163520 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f55ac0019f0 con 0x7f55b4163520 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 >> v1:172.21.15.150:6789/0 conn(0x7f55b4163520 legacy=0x7f55b415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55980039e0 0x7f5598003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598004800 0x7f5598004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f559800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55b40afde0 con 0x7f5598005210 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f559800c140 con 0x7f5598004800 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.133+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f559800c340 con 0x7f55980039e0 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b2800640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f559800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b1400640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598004800 0x7f5598004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b1e00640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55980039e0 0x7f5598003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b2800640 1 -- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55980039e0 msgr2=0x7f5598003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:59.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b2800640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55980039e0 0x7f5598003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b2800640 1 -- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598004800 msgr2=0x7f5598004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:59.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b2800640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598004800 0x7f5598004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b2800640 1 -- 172.21.15.150:0/321254884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5598003520 con 0x7f5598005210 2024-04-15T15:25:59.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.134+0000 7f55b1e00640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55980039e0 0x7f5598003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:59.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.135+0000 7f55b2800640 1 -- 172.21.15.150:0/321254884 reap_dead start 2024-04-15T15:25:59.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.135+0000 7f55b2800640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f559800b9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00a810 tx=0x7f5598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca91d628aec2017b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:59.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00a460 con 0x7f5598005210 2024-04-15T15:25:59.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac034650 con 0x7f5598005210 2024-04-15T15:25:59.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b0a00640 1 -- 172.21.15.150:0/321254884 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac034a40 con 0x7f5598005210 2024-04-15T15:25:59.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b8c00640 1 -- 172.21.15.150:0/321254884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 msgr2=0x7f559800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b8c00640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f559800b9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00a810 tx=0x7f5598005830 comp rx=0 tx=0).stop 2024-04-15T15:25:59.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b8c00640 1 -- 172.21.15.150:0/321254884 shutdown_connections 2024-04-15T15:25:59.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b8c00640 1 --2- 172.21.15.150:0/321254884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f559800b9c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b8c00640 1 -- 172.21.15.150:0/321254884 >> 172.21.15.150:0/321254884 conn(0x7f55b40b2460 msgr2=0x7f55b40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:59.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b8c00640 1 -- 172.21.15.150:0/321254884 shutdown_connections 2024-04-15T15:25:59.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.136+0000 7f55b8c00640 1 -- 172.21.15.150:0/321254884 wait complete. 2024-04-15T15:25:59.141 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.138+0000 7f55b8c00640 1 Processor -- start 2024-04-15T15:25:59.141 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.138+0000 7f55b8c00640 1 -- start start 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b8c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f55b406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b8c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b41421c0 0x7f55b406c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415c8d0 0x7f55b41401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b8c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4145450 con 0x7f5598005210 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b8c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55b41627f0 con 0x7f55b41421c0 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55b4072150 con 0x7f55b415c8d0 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f55b406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f55b406bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50970/0 (socket says 172.21.15.150:50970) 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.139+0000 7f55b1e00640 1 -- 172.21.15.150:0/3383349337 learned_addr learned my addr 172.21.15.150:0/3383349337 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b1400640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b41421c0 0x7f55b406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b1e00640 1 -- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415c8d0 msgr2=0x7f55b41401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b1e00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415c8d0 0x7f55b41401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b1e00640 1 -- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b41421c0 msgr2=0x7f55b406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b1e00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b41421c0 0x7f55b406c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b1e00640 1 -- 172.21.15.150:0/3383349337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b4161760 con 0x7f5598005210 2024-04-15T15:25:59.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b1400640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b41421c0 0x7f55b406c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:59.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b1e00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f55b406bcf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f55a400d560 tx=0x7f55a400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:59.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55aa000640 1 -- 172.21.15.150:0/3383349337 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4005480 con 0x7f5598005210 2024-04-15T15:25:59.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55aa000640 1 -- 172.21.15.150:0/3383349337 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a4017070 con 0x7f5598005210 2024-04-15T15:25:59.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55aa000640 1 -- 172.21.15.150:0/3383349337 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4003810 con 0x7f5598005210 2024-04-15T15:25:59.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.140+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b4161a40 con 0x7f5598005210 2024-04-15T15:25:59.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.141+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b4162000 con 0x7f5598005210 2024-04-15T15:25:59.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.142+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b4068c80 con 0x7f5598005210 2024-04-15T15:25:59.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.143+0000 7f55aa000640 1 -- 172.21.15.150:0/3383349337 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f55a40039b0 con 0x7f5598005210 2024-04-15T15:25:59.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.144+0000 7f55aa000640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f559006c980 0x7f559006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.144+0000 7f55aa000640 1 -- 172.21.15.150:0/3383349337 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f55a408b810 con 0x7f5598005210 2024-04-15T15:25:59.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.145+0000 7f55b1400640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f559006c980 0x7f559006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.146+0000 7f55b1400640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f559006c980 0x7f559006ee40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f55b41425c0 tx=0x7f559c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:59.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.149+0000 7f55aa000640 1 -- 172.21.15.150:0/3383349337 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55a4019310 con 0x7f5598005210 2024-04-15T15:25:59.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.351+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f55b41430d0 con 0x7f5598005210 2024-04-15T15:25:59.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.358+0000 7f55aa000640 1 -- 172.21.15.150:0/3383349337 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) ==== 104+0+0 (secure 0 0 0) 0x7f55a4059e70 con 0x7f5598005210 2024-04-15T15:25:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f559006c980 msgr2=0x7f559006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f559006c980 0x7f559006ee40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f55b41425c0 tx=0x7f559c008040 comp rx=0 tx=0).stop 2024-04-15T15:25:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 msgr2=0x7f55b406bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f55b406bcf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f55a400d560 tx=0x7f55a400da20 comp rx=0 tx=0).stop 2024-04-15T15:25:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 shutdown_connections 2024-04-15T15:25:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f559006c980 0x7f559006ee40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415c8d0 0x7f55b41401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b41421c0 0x7f55b406c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 --2- 172.21.15.150:0/3383349337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598005210 0x7f55b406bcf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.361+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 >> 172.21.15.150:0/3383349337 conn(0x7f55b40b2460 msgr2=0x7f55b40cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.362+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 shutdown_connections 2024-04-15T15:25:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.362+0000 7f55b8c00640 1 -- 172.21.15.150:0/3383349337 wait complete. 2024-04-15T15:25:59.382 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds debug_mds 2024-04-15T15:25:59.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.589+0000 7fbbc4c00640 1 Processor -- start 2024-04-15T15:25:59.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.589+0000 7fbbc4c00640 1 -- start start 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.591+0000 7fbbc4c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc01461c0 0x7fbbc0071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.591+0000 7fbbc4c00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbbc00cebb0 0x7fbbc006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.591+0000 7fbbc4c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc00cd2d0 con 0x7fbbc0072420 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.591+0000 7fbbc4c00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fbbc00cf1d0 con 0x7fbbc00cebb0 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.591+0000 7fbbc4c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc0072880 con 0x7fbbc01461c0 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.592+0000 7fbbbfe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbbc00cebb0 0x7fbbc006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.592+0000 7fbbbfe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbbc00cebb0 0x7fbbc006df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50976/0 (socket says 172.21.15.150:50976) 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.592+0000 7fbbbfe00640 1 -- 172.21.15.150:0/4005346838 learned_addr learned my addr 172.21.15.150:0/4005346838 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.592+0000 7fbbbf400640 1 --2- 172.21.15.150:0/4005346838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc01461c0 0x7fbbc0071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.592+0000 7fbbbfe00640 1 -- 172.21.15.150:0/4005346838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc01461c0 msgr2=0x7fbbc0071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:59.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.592+0000 7fbbbfe00640 1 --2- 172.21.15.150:0/4005346838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc01461c0 0x7fbbc0071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.592+0000 7fbbbfe00640 1 -- 172.21.15.150:0/4005346838 >> v1:172.21.15.187:6789/0 conn(0x7fbbc0072420 legacy=0x7fbbc00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:25:59.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.592+0000 7fbbbfe00640 1 -- 172.21.15.150:0/4005346838 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc00cd5b0 con 0x7fbbc00cebb0 2024-04-15T15:25:59.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.593+0000 7fbbbfe00640 1 --2- 172.21.15.150:0/4005346838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc006df20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009e50 tx=0x7fbbb802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7248ad5c39ebef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:59.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbbea00640 1 -- 172.21.15.150:0/4005346838 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb802ca70 con 0x7fbbc00cebb0 2024-04-15T15:25:59.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbbea00640 1 -- 172.21.15.150:0/4005346838 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb802cc10 con 0x7fbbc00cebb0 2024-04-15T15:25:59.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbbea00640 1 -- 172.21.15.150:0/4005346838 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb80343f0 con 0x7fbbc00cebb0 2024-04-15T15:25:59.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbc4c00640 1 -- 172.21.15.150:0/4005346838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 msgr2=0x7fbbc006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/4005346838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc006df20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009e50 tx=0x7fbbb802bed0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbc4c00640 1 -- 172.21.15.150:0/4005346838 shutdown_connections 2024-04-15T15:25:59.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/4005346838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc006df20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/4005346838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc01461c0 0x7fbbc0071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbc4c00640 1 -- 172.21.15.150:0/4005346838 >> 172.21.15.150:0/4005346838 conn(0x7fbbc006b930 msgr2=0x7fbbc006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:59.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.594+0000 7fbbc4c00640 1 -- 172.21.15.150:0/4005346838 shutdown_connections 2024-04-15T15:25:59.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.595+0000 7fbbc4c00640 1 -- 172.21.15.150:0/4005346838 wait complete. 2024-04-15T15:25:59.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.596+0000 7fbbc4c00640 1 Processor -- start 2024-04-15T15:25:59.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.596+0000 7fbbc4c00640 1 -- start start 2024-04-15T15:25:59.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.597+0000 7fbbc4c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc0072420 0x7fbbc00a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.597+0000 7fbbc4c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc00aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.597+0000 7fbbc4c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc01461c0 0x7fbbc00aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.597+0000 7fbbc4c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc00cdb70 con 0x7fbbc00cebb0 2024-04-15T15:25:59.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.597+0000 7fbbc4c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc00cd080 con 0x7fbbc01461c0 2024-04-15T15:25:59.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.597+0000 7fbbc4c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc006e460 con 0x7fbbc0072420 2024-04-15T15:25:59.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbb6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc00aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbb6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc00aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50978/0 (socket says 172.21.15.150:50978) 2024-04-15T15:25:59.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbb6a00640 1 -- 172.21.15.150:0/2163431444 learned_addr learned my addr 172.21.15.150:0/2163431444 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:25:59.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbbf400640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc0072420 0x7fbbc00a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbbfe00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc01461c0 0x7fbbc00aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbb6a00640 1 -- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc0072420 msgr2=0x7fbbc00a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbb6a00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc0072420 0x7fbbc00a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbb6a00640 1 -- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc01461c0 msgr2=0x7fbbc00aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbb6a00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc01461c0 0x7fbbc00aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbb6a00640 1 -- 172.21.15.150:0/2163431444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc00aa750 con 0x7fbbc00cebb0 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbbf400640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc0072420 0x7fbbc00a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.598+0000 7fbbbfe00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc01461c0 0x7fbbc00aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.599+0000 7fbbb6a00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc00aed40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbba800c430 tx=0x7fbba800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.599+0000 7fbbb7e00640 1 -- 172.21.15.150:0/2163431444 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8011790 con 0x7fbbc00cebb0 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.599+0000 7fbbb7e00640 1 -- 172.21.15.150:0/2163431444 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba8017070 con 0x7fbbc00cebb0 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.599+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc00aac40 con 0x7fbbc00cebb0 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.599+0000 7fbbb7e00640 1 -- 172.21.15.150:0/2163431444 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8011bb0 con 0x7fbbc00cebb0 2024-04-15T15:25:59.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.599+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc00ab200 con 0x7fbbc00cebb0 2024-04-15T15:25:59.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.600+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc00755c0 con 0x7fbbc00cebb0 2024-04-15T15:25:59.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.601+0000 7fbbb7e00640 1 -- 172.21.15.150:0/2163431444 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fbba801c460 con 0x7fbbc00cebb0 2024-04-15T15:25:59.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.602+0000 7fbbb7e00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbb9c06c980 0x7fbb9c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:25:59.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.602+0000 7fbbb7e00640 1 -- 172.21.15.150:0/2163431444 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fbba808b900 con 0x7fbbc00cebb0 2024-04-15T15:25:59.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.603+0000 7fbbbf400640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbb9c06c980 0x7fbb9c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:25:59.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.604+0000 7fbbbf400640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbb9c06c980 0x7fbb9c06ee40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0006fd0 tx=0x7fbbb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:25:59.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.607+0000 7fbbb7e00640 1 -- 172.21.15.150:0/2163431444 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbba8059ee0 con 0x7fbbc00cebb0 2024-04-15T15:25:59.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.807+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "debug_mds"} v 0) -- 0x7fbbc0072e40 con 0x7fbbc00cebb0 2024-04-15T15:25:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.813+0000 7fbbb7e00640 1 -- 172.21.15.150:0/2163431444 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "debug_mds"}]=0 v15) ==== 92+0+0 (secure 0 0 0) 0x7fbba80597e0 con 0x7fbbc00cebb0 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbb9c06c980 msgr2=0x7fbb9c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbb9c06c980 0x7fbb9c06ee40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0006fd0 tx=0x7fbbb0008040 comp rx=0 tx=0).stop 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 msgr2=0x7fbbc00aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc00aed40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbba800c430 tx=0x7fbba800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 shutdown_connections 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc01461c0 0x7fbbc00aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbc00cebb0 0x7fbbc00aed40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fbb9c06c980 0x7fbb9c06ee40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 --2- 172.21.15.150:0/2163431444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbc0072420 0x7fbbc00a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:25:59.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 >> 172.21.15.150:0/2163431444 conn(0x7fbbc006b930 msgr2=0x7fbbc006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:25:59.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 shutdown_connections 2024-04-15T15:25:59.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:25:59.816+0000 7fbbc4c00640 1 -- 172.21.15.150:0/2163431444 wait complete. 2024-04-15T15:25:59.838 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked' 2024-04-15T15:26:00.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.052+0000 7fccaf200640 1 Processor -- start 2024-04-15T15:26:00.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.052+0000 7fccaf200640 1 -- start start 2024-04-15T15:26:00.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.053+0000 7fccaf200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb00cee00 0x7fccb006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:00.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.053+0000 7fccaf200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fccb015d2f0 0x7fccb00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:00.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.053+0000 7fccaf200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb015dbb0 con 0x7fccb0071f50 2024-04-15T15:26:00.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.053+0000 7fccaf200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fccb006f4f0 con 0x7fccb015d2f0 2024-04-15T15:26:00.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.053+0000 7fccaf200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fccb00728f0 con 0x7fccb00cee00 2024-04-15T15:26:00.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.054+0000 7fccae800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fccb015d2f0 0x7fccb00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:00.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.054+0000 7fccade00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb00cee00 0x7fccb006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:00.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.054+0000 7fccad400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fccb0071f50 0x7fccb015cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:52116/0 (socket says 172.21.15.150:52116) 2024-04-15T15:26:00.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.054+0000 7fccad400640 1 -- 172.21.15.150:0/3766926316 learned_addr learned my addr 172.21.15.150:0/3766926316 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:00.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.055+0000 7fccae800640 1 -- 172.21.15.150:0/3766926316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb00cee00 msgr2=0x7fccb006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:00.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.055+0000 7fccae800640 1 --2- 172.21.15.150:0/3766926316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb00cee00 0x7fccb006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.055+0000 7fccae800640 1 -- 172.21.15.150:0/3766926316 >> v1:172.21.15.187:6789/0 conn(0x7fccb0071f50 legacy=0x7fccb015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:00.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.055+0000 7fccae800640 1 -- 172.21.15.150:0/3766926316 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb015d8d0 con 0x7fccb015d2f0 2024-04-15T15:26:00.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.056+0000 7fccae800640 1 --2- 172.21.15.150:0/3766926316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb015d2f0 0x7fccb00723b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009dc0 tx=0x7fcca802be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2d570ea2aa441b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:00.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaca00640 1 -- 172.21.15.150:0/3766926316 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8039070 con 0x7fccb015d2f0 2024-04-15T15:26:00.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaca00640 1 -- 172.21.15.150:0/3766926316 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca802cc30 con 0x7fccb015d2f0 2024-04-15T15:26:00.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaca00640 1 -- 172.21.15.150:0/3766926316 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8034440 con 0x7fccb015d2f0 2024-04-15T15:26:00.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaf200640 1 -- 172.21.15.150:0/3766926316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb015d2f0 msgr2=0x7fccb00723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:00.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaf200640 1 --2- 172.21.15.150:0/3766926316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb015d2f0 0x7fccb00723b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009dc0 tx=0x7fcca802be80 comp rx=0 tx=0).stop 2024-04-15T15:26:00.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaf200640 1 -- 172.21.15.150:0/3766926316 shutdown_connections 2024-04-15T15:26:00.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaf200640 1 --2- 172.21.15.150:0/3766926316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb015d2f0 0x7fccb00723b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaf200640 1 --2- 172.21.15.150:0/3766926316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb00cee00 0x7fccb006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.057+0000 7fccaf200640 1 -- 172.21.15.150:0/3766926316 >> 172.21.15.150:0/3766926316 conn(0x7fccb006b930 msgr2=0x7fccb006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:00.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.058+0000 7fccaf200640 1 -- 172.21.15.150:0/3766926316 shutdown_connections 2024-04-15T15:26:00.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.058+0000 7fccaf200640 1 -- 172.21.15.150:0/3766926316 wait complete. 2024-04-15T15:26:00.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.059+0000 7fccaf200640 1 Processor -- start 2024-04-15T15:26:00.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.059+0000 7fccaf200640 1 -- start start 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.060+0000 7fccaf200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0071f50 0x7fccb01e6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.060+0000 7fccaf200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb00cee00 0x7fccb0159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.060+0000 7fccaf200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb015d2f0 0x7fccb015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.060+0000 7fccaf200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccb006e3a0 con 0x7fccb0071f50 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.060+0000 7fccaf200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fccb006e0c0 con 0x7fccb00cee00 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.060+0000 7fccaf200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fccb015dda0 con 0x7fccb015d2f0 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccade00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0071f50 0x7fccb01e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccade00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0071f50 0x7fccb01e6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50986/0 (socket says 172.21.15.150:50986) 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccade00640 1 -- 172.21.15.150:0/3524636943 learned_addr learned my addr 172.21.15.150:0/3524636943 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccad400640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb00cee00 0x7fccb0159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccae800640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb015d2f0 0x7fccb015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:00.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccade00640 1 -- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb015d2f0 msgr2=0x7fccb015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:00.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccade00640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb015d2f0 0x7fccb015f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccade00640 1 -- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb00cee00 msgr2=0x7fccb0159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:00.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccade00640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb00cee00 0x7fccb0159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.061+0000 7fccade00640 1 -- 172.21.15.150:0/3524636943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb01586a0 con 0x7fccb0071f50 2024-04-15T15:26:00.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.062+0000 7fccade00640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0071f50 0x7fccb01e6010 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e9e0 tx=0x7fcca000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:00.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.062+0000 7fcca6000640 1 -- 172.21.15.150:0/3524636943 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0017070 con 0x7fccb0071f50 2024-04-15T15:26:00.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.062+0000 7fcca6000640 1 -- 172.21.15.150:0/3524636943 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca000c970 con 0x7fccb0071f50 2024-04-15T15:26:00.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.062+0000 7fcca6000640 1 -- 172.21.15.150:0/3524636943 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca000cd60 con 0x7fccb0071f50 2024-04-15T15:26:00.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.062+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb01e6ac0 con 0x7fccb0071f50 2024-04-15T15:26:00.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.062+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb01e7080 con 0x7fccb0071f50 2024-04-15T15:26:00.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.063+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb00755c0 con 0x7fccb0071f50 2024-04-15T15:26:00.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.064+0000 7fcca6000640 1 -- 172.21.15.150:0/3524636943 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcca001a070 con 0x7fccb0071f50 2024-04-15T15:26:00.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.065+0000 7fcca6000640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc9006c980 0x7fcc9006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:00.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.065+0000 7fcca6000640 1 -- 172.21.15.150:0/3524636943 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fcca008bc50 con 0x7fccb0071f50 2024-04-15T15:26:00.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.066+0000 7fccad400640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc9006c980 0x7fcc9006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:00.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.067+0000 7fccad400640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc9006c980 0x7fcc9006ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fccb01e6550 tx=0x7fcc9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:00.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.070+0000 7fcca6000640 1 -- 172.21.15.150:0/3524636943 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcca005a330 con 0x7fccb0071f50 2024-04-15T15:26:00.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.347+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]} v 0) -- 0x7fccb006f970 con 0x7fccb0071f50 2024-04-15T15:26:00.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.450+0000 7fcca6000640 1 -- 172.21.15.150:0/3524636943 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]}]=0 v137) ==== 137+0+0 (secure 0 0 0) 0x7fcca0059c30 con 0x7fccb0071f50 2024-04-15T15:26:00.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc9006c980 msgr2=0x7fcc9006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:00.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc9006c980 0x7fcc9006ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fccb01e6550 tx=0x7fcc9c009040 comp rx=0 tx=0).stop 2024-04-15T15:26:00.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0071f50 msgr2=0x7fccb01e6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:00.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0071f50 0x7fccb01e6010 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e9e0 tx=0x7fcca000eea0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 shutdown_connections 2024-04-15T15:26:00.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcc9006c980 0x7fcc9006ee40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fccb015d2f0 0x7fccb015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb00cee00 0x7fccb0159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 --2- 172.21.15.150:0/3524636943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0071f50 0x7fccb01e6010 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:00.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 >> 172.21.15.150:0/3524636943 conn(0x7fccb006b930 msgr2=0x7fccb00b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:00.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 shutdown_connections 2024-04-15T15:26:00.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:00.453+0000 7fccaf200640 1 -- 172.21.15.150:0/3524636943 wait complete. 2024-04-15T15:26:00.476 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. ... FAIL 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner: 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner:FAIL: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 541, in test_quiesce_blocked 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner: self._verify_quiesce(root=self.subvolume) 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 138, in _verify_quiesce 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner: return self._verify_quiesce_wrapped(rank, status, root, root_inode, ops, cache, splitauth) 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 272, in _verify_quiesce_wrapped 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner: self.assertFalse(lock_type.startswith("i")) 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner:AssertionError: True is not false 2024-04-15T15:26:00.477 INFO:tasks.cephfs_test_runner: 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner:Ran 1 test in 84.921s 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner: 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner: 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner:FAIL: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 541, in test_quiesce_blocked 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner: self._verify_quiesce(root=self.subvolume) 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 138, in _verify_quiesce 2024-04-15T15:26:00.478 INFO:tasks.cephfs_test_runner: return self._verify_quiesce_wrapped(rank, status, root, root_inode, ops, cache, splitauth) 2024-04-15T15:26:00.479 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs/test_quiesce.py", line 272, in _verify_quiesce_wrapped 2024-04-15T15:26:00.479 INFO:tasks.cephfs_test_runner: self.assertFalse(lock_type.startswith("i")) 2024-04-15T15:26:00.479 INFO:tasks.cephfs_test_runner:AssertionError: True is not false 2024-04-15T15:26:00.479 INFO:tasks.cephfs_test_runner: 2024-04-15T15:26:00.479 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:26:00.725 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=bb02693e2f3542b282d209b8830efe97 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9bf1c4ba967dd24077be6e5b87edda4ca87a6e14/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-15T15:26:00.727 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-15T15:26:00.736 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-15T15:26:00.749 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-15T15:26:00.749 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:26:00.749 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:26:00.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052773,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:26:00.809 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-15T15:26:00.809 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:26:00.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052773,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-15T15:26:00.864 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:26:00.864 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:26:00.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052761,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:26:00.922 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-15T15:26:00.922 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-15T15:26:00.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052761,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-15T15:26:00.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:00.975 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:26:01.034 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-15T15:26:01.035 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-15T15:26:01.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.036 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-15T15:26:01.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.138 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-15T15:26:01.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.222 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-04-15T15:26:01.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.365 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-15T15:26:01.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.408 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-04-15T15:26:01.530 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-04-15T15:26:01.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:26:01.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.534 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:26:01.591 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-15T15:26:01.592 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-15T15:26:01.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.592 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-15T15:26:01.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.682 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-15T15:26:01.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.762 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-04-15T15:26:01.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.895 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-04-15T15:26:01.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-15T15:26:01.939 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-04-15T15:26:02.076 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-04-15T15:26:02.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:26:02.080 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-15T15:26:02.090 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-15T15:26:02.090 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:26:02.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.309+0000 7f348c200640 1 Processor -- start 2024-04-15T15:26:02.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.309+0000 7f348c200640 1 -- start start 2024-04-15T15:26:02.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.311+0000 7f348c200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 0x7f34840b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.311+0000 7f348c200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34840b3e10 0x7f348415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.311+0000 7f348c200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34840b4be0 con 0x7f34841461f0 2024-04-15T15:26:02.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.311+0000 7f348c200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f34840a9640 con 0x7f34840b3e10 2024-04-15T15:26:02.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.311+0000 7f348c200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f34840b4620 con 0x7f34840cefc0 2024-04-15T15:26:02.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.311+0000 7f348a200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34840b3e10 0x7f348415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.311+0000 7f3489800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f34841461f0 0x7f34840ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38366/0 (socket says 172.21.15.150:38366) 2024-04-15T15:26:02.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.311+0000 7f3489800640 1 -- 172.21.15.150:0/948884337 learned_addr learned my addr 172.21.15.150:0/948884337 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:02.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.312+0000 7f3488e00640 1 --2- 172.21.15.150:0/948884337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 0x7f34840b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.312+0000 7f348a200640 1 -- 172.21.15.150:0/948884337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 msgr2=0x7f34840b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:02.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.312+0000 7f348a200640 1 --2- 172.21.15.150:0/948884337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 0x7f34840b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.312+0000 7f348a200640 1 -- 172.21.15.150:0/948884337 >> v1:172.21.15.150:6789/0 conn(0x7f34841461f0 legacy=0x7f34840ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.312+0000 7f348a200640 1 -- 172.21.15.150:0/948884337 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34840b4900 con 0x7f34840b3e10 2024-04-15T15:26:02.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.313+0000 7f348a200640 1 --2- 172.21.15.150:0/948884337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f348415d7b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f347c0097d0 tx=0x7f347c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfbcbf43a297ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:02.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.313+0000 7f3483e00640 1 -- 172.21.15.150:0/948884337 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c035020 con 0x7f34840b3e10 2024-04-15T15:26:02.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.313+0000 7f3483e00640 1 -- 172.21.15.150:0/948884337 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347c031810 con 0x7f34840b3e10 2024-04-15T15:26:02.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.313+0000 7f3483e00640 1 -- 172.21.15.150:0/948884337 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c031be0 con 0x7f34840b3e10 2024-04-15T15:26:02.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.314+0000 7f348c200640 1 -- 172.21.15.150:0/948884337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 msgr2=0x7f348415d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.314+0000 7f348c200640 1 --2- 172.21.15.150:0/948884337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f348415d7b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f347c0097d0 tx=0x7f347c02b8b0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.314+0000 7f348c200640 1 -- 172.21.15.150:0/948884337 shutdown_connections 2024-04-15T15:26:02.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.314+0000 7f348c200640 1 --2- 172.21.15.150:0/948884337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 0x7f34840b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.314+0000 7f348c200640 1 --2- 172.21.15.150:0/948884337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f348415d7b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.314+0000 7f348c200640 1 -- 172.21.15.150:0/948884337 >> 172.21.15.150:0/948884337 conn(0x7f34840b27b0 msgr2=0x7f34840b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:02.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.314+0000 7f348c200640 1 -- 172.21.15.150:0/948884337 shutdown_connections 2024-04-15T15:26:02.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.314+0000 7f348c200640 1 -- 172.21.15.150:0/948884337 wait complete. 2024-04-15T15:26:02.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.316+0000 7f348c200640 1 Processor -- start 2024-04-15T15:26:02.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.316+0000 7f348c200640 1 -- start start 2024-04-15T15:26:02.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f348c200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f34841f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f348c200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 0x7f34841f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f348c200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34841461f0 0x7f34840aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f348c200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34841465d0 con 0x7f34840b3e10 2024-04-15T15:26:02.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f348c200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f348415d290 con 0x7f34841461f0 2024-04-15T15:26:02.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f348c200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f34840b42a0 con 0x7f34840cefc0 2024-04-15T15:26:02.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f3489800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f34841f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f3489800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f34841f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51010/0 (socket says 172.21.15.150:51010) 2024-04-15T15:26:02.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.317+0000 7f3489800640 1 -- 172.21.15.150:0/3545653463 learned_addr learned my addr 172.21.15.150:0/3545653463 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:02.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f348a200640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34841461f0 0x7f34840aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3488e00640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 0x7f34841f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3489800640 1 -- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 msgr2=0x7f34841f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3489800640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 0x7f34841f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3489800640 1 -- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34841461f0 msgr2=0x7f34840aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3489800640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34841461f0 0x7f34840aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3489800640 1 -- 172.21.15.150:0/3545653463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3484070200 con 0x7f34840b3e10 2024-04-15T15:26:02.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f348a200640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34841461f0 0x7f34840aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:02.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3489800640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f34841f7750 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f347800ca70 tx=0x7f347800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:02.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3481600640 1 -- 172.21.15.150:0/3545653463 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478017070 con 0x7f34840b3e10 2024-04-15T15:26:02.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3481600640 1 -- 172.21.15.150:0/3545653463 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347800abd0 con 0x7f34840b3e10 2024-04-15T15:26:02.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34841e70d0 con 0x7f34840b3e10 2024-04-15T15:26:02.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f3481600640 1 -- 172.21.15.150:0/3545653463 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478012400 con 0x7f34840b3e10 2024-04-15T15:26:02.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.318+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34841e7690 con 0x7f34840b3e10 2024-04-15T15:26:02.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.320+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3484068c80 con 0x7f34840b3e10 2024-04-15T15:26:02.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.321+0000 7f3481600640 1 -- 172.21.15.150:0/3545653463 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f347801a070 con 0x7f34840b3e10 2024-04-15T15:26:02.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.322+0000 7f3481600640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f346406c980 0x7f346406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.322+0000 7f3481600640 1 -- 172.21.15.150:0/3545653463 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f347808b700 con 0x7f34840b3e10 2024-04-15T15:26:02.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.324+0000 7f3488e00640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f346406c980 0x7f346406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.325+0000 7f3488e00640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f346406c980 0x7f346406ee40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f34840afeb0 tx=0x7f3470008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:02.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.328+0000 7f3481600640 1 -- 172.21.15.150:0/3545653463 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3478059ce0 con 0x7f34840b3e10 2024-04-15T15:26:02.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.526+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34840cf440 con 0x7f346406c980 2024-04-15T15:26:02.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.528+0000 7f3481600640 1 -- 172.21.15.150:0/3545653463 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13497 (secure 0 0 0) 0x7f34840cf440 con 0x7f346406c980 2024-04-15T15:26:02.529 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:26:02.530 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:26:02.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f346406c980 msgr2=0x7f346406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f346406c980 0x7f346406ee40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f34840afeb0 tx=0x7f3470008040 comp rx=0 tx=0).stop 2024-04-15T15:26:02.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 msgr2=0x7f34841f7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f34841f7750 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f347800ca70 tx=0x7f347800cf30 comp rx=0 tx=0).stop 2024-04-15T15:26:02.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 shutdown_connections 2024-04-15T15:26:02.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34841461f0 0x7f34840aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f346406c980 0x7f346406ee40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cefc0 0x7f34841f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 --2- 172.21.15.150:0/3545653463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34840b3e10 0x7f34841f7750 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 >> 172.21.15.150:0/3545653463 conn(0x7f34840b27b0 msgr2=0x7f34840b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:02.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.531+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 shutdown_connections 2024-04-15T15:26:02.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.532+0000 7f348c200640 1 -- 172.21.15.150:0/3545653463 wait complete. 2024-04-15T15:26:02.551 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-15T15:26:01.261312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224972,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749748,"statfs":{"total":773094113280,"available":772863741952,"internally_reserved":0,"allocated":4759552,"data_stored":2676452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-74746,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-132,"num_write":-106,"num_write_kb":-118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002106"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-15T15:25:57.621874+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:25:57.621874+0000","last_peered":"2024-04-15T15:25:57.621874+0000","last_clean":"2024-04-15T15:25:57.621874+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:25:57.621874+0000","last_undegraded":"2024-04-15T15:25:57.621874+0000","last_fullsized":"2024-04-15T15:25:57.621874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":755012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":165170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":170643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":164658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":247534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":247667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1028096,"data_stored":761144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:26:02.551 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:26:02.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.750+0000 7fad28a00640 1 Processor -- start 2024-04-15T15:26:02.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.750+0000 7fad28a00640 1 -- start start 2024-04-15T15:26:02.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad28a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240d1c90 0x7fad24162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad28a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad240cf110 0x7fad240d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad28a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad240d0090 con 0x7fad24162020 2024-04-15T15:26:02.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad28a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fad24163440 con 0x7fad240cf110 2024-04-15T15:26:02.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad28a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad240b7b30 con 0x7fad240d1c90 2024-04-15T15:26:02.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad23200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad240cf110 0x7fad240d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad23200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad240cf110 0x7fad240d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51018/0 (socket says 172.21.15.150:51018) 2024-04-15T15:26:02.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad23200640 1 -- 172.21.15.150:0/3479393490 learned_addr learned my addr 172.21.15.150:0/3479393490 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:02.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.752+0000 7fad19e00640 1 --2- 172.21.15.150:0/3479393490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240d1c90 0x7fad24162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.753+0000 7fad23200640 1 -- 172.21.15.150:0/3479393490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240d1c90 msgr2=0x7fad24162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:02.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.753+0000 7fad23200640 1 --2- 172.21.15.150:0/3479393490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240d1c90 0x7fad24162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.753+0000 7fad23200640 1 -- 172.21.15.150:0/3479393490 >> v1:172.21.15.187:6789/0 conn(0x7fad24162020 legacy=0x7fad240d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:02.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.753+0000 7fad23200640 1 -- 172.21.15.150:0/3479393490 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad240cfad0 con 0x7fad240cf110 2024-04-15T15:26:02.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.754+0000 7fad23200640 1 --2- 172.21.15.150:0/3479393490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240cf110 0x7fad240d3390 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009e50 tx=0x7fad1c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6d35c584175414 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:02.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.754+0000 7fad21e00640 1 -- 172.21.15.150:0/3479393490 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c02ca70 con 0x7fad240cf110 2024-04-15T15:26:02.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.754+0000 7fad21e00640 1 -- 172.21.15.150:0/3479393490 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1c02cc10 con 0x7fad240cf110 2024-04-15T15:26:02.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.754+0000 7fad21e00640 1 -- 172.21.15.150:0/3479393490 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0343f0 con 0x7fad240cf110 2024-04-15T15:26:02.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.755+0000 7fad28a00640 1 -- 172.21.15.150:0/3479393490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240cf110 msgr2=0x7fad240d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.755+0000 7fad28a00640 1 --2- 172.21.15.150:0/3479393490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240cf110 0x7fad240d3390 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009e50 tx=0x7fad1c02bed0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.755+0000 7fad28a00640 1 -- 172.21.15.150:0/3479393490 shutdown_connections 2024-04-15T15:26:02.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.755+0000 7fad28a00640 1 --2- 172.21.15.150:0/3479393490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240cf110 0x7fad240d3390 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.755+0000 7fad28a00640 1 --2- 172.21.15.150:0/3479393490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240d1c90 0x7fad24162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.755+0000 7fad28a00640 1 -- 172.21.15.150:0/3479393490 >> 172.21.15.150:0/3479393490 conn(0x7fad2406cf00 msgr2=0x7fad2406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:02.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.755+0000 7fad28a00640 1 -- 172.21.15.150:0/3479393490 shutdown_connections 2024-04-15T15:26:02.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.755+0000 7fad28a00640 1 -- 172.21.15.150:0/3479393490 wait complete. 2024-04-15T15:26:02.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.757+0000 7fad28a00640 1 Processor -- start 2024-04-15T15:26:02.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.757+0000 7fad28a00640 1 -- start start 2024-04-15T15:26:02.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad28a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240cf110 0x7fad2414ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad28a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240d1c90 0x7fad24154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad28a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad24162020 0x7fad24154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad28a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad240d2b90 con 0x7fad240d1c90 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad28a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fad240cf880 con 0x7fad24162020 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad28a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad240d2e70 con 0x7fad240cf110 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad19e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240d1c90 0x7fad24154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad19e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240d1c90 0x7fad24154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51024/0 (socket says 172.21.15.150:51024) 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad19e00640 1 -- 172.21.15.150:0/3438485753 learned_addr learned my addr 172.21.15.150:0/3438485753 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad23200640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad24162020 0x7fad24154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad22800640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240cf110 0x7fad2414ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad19e00640 1 -- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240cf110 msgr2=0x7fad2414ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad19e00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240cf110 0x7fad2414ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad19e00640 1 -- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad24162020 msgr2=0x7fad24154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad19e00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad24162020 0x7fad24154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.758+0000 7fad19e00640 1 -- 172.21.15.150:0/3438485753 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2414f230 con 0x7fad240d1c90 2024-04-15T15:26:02.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.759+0000 7fad23200640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad24162020 0x7fad24154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:02.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.759+0000 7fad19e00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240d1c90 0x7fad24154a00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00d5c0 tx=0x7fad0c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:02.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.759+0000 7fad1b400640 1 -- 172.21.15.150:0/3438485753 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0125f0 con 0x7fad240d1c90 2024-04-15T15:26:02.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.759+0000 7fad1b400640 1 -- 172.21.15.150:0/3438485753 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c019070 con 0x7fad240d1c90 2024-04-15T15:26:02.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.759+0000 7fad1b400640 1 -- 172.21.15.150:0/3438485753 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0129d0 con 0x7fad240d1c90 2024-04-15T15:26:02.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.759+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad240cd7f0 con 0x7fad240d1c90 2024-04-15T15:26:02.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.759+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad240cddb0 con 0x7fad240d1c90 2024-04-15T15:26:02.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.760+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad24161560 con 0x7fad240d1c90 2024-04-15T15:26:02.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.762+0000 7fad1b400640 1 -- 172.21.15.150:0/3438485753 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fad0c012b70 con 0x7fad240d1c90 2024-04-15T15:26:02.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.763+0000 7fad1b400640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fad0006c980 0x7fad0006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:02.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.763+0000 7fad1b400640 1 -- 172.21.15.150:0/3438485753 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fad0c08bd60 con 0x7fad240d1c90 2024-04-15T15:26:02.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.764+0000 7fad22800640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fad0006c980 0x7fad0006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:02.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.765+0000 7fad22800640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fad0006c980 0x7fad0006ee40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fad14003d90 tx=0x7fad14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:02.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.767+0000 7fad1b400640 1 -- 172.21.15.150:0/3438485753 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad24161560 con 0x7fad240d1c90 2024-04-15T15:26:02.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.951+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad240755c0 con 0x7fad0006c980 2024-04-15T15:26:02.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.954+0000 7fad1b400640 1 -- 172.21.15.150:0/3438485753 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13497 (secure 0 0 0) 0x7fad240755c0 con 0x7fad0006c980 2024-04-15T15:26:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:26:02.957 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:26:02.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fad0006c980 msgr2=0x7fad0006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fad0006c980 0x7fad0006ee40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fad14003d90 tx=0x7fad14009040 comp rx=0 tx=0).stop 2024-04-15T15:26:02.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240d1c90 msgr2=0x7fad24154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:02.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240d1c90 0x7fad24154a00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00d5c0 tx=0x7fad0c00da80 comp rx=0 tx=0).stop 2024-04-15T15:26:02.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 shutdown_connections 2024-04-15T15:26:02.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad24162020 0x7fad24154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad240d1c90 0x7fad24154a00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fad0006c980 0x7fad0006ee40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 --2- 172.21.15.150:0/3438485753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad240cf110 0x7fad2414ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:02.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 >> 172.21.15.150:0/3438485753 conn(0x7fad2406cf00 msgr2=0x7fad2415f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:02.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 shutdown_connections 2024-04-15T15:26:02.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:02.957+0000 7fad28a00640 1 -- 172.21.15.150:0/3438485753 wait complete. 2024-04-15T15:26:02.977 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-15T15:26:01.261312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224972,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749748,"statfs":{"total":773094113280,"available":772863741952,"internally_reserved":0,"allocated":4759552,"data_stored":2676452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-74746,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-132,"num_write":-106,"num_write_kb":-118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002106"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-15T15:25:57.621874+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:25:57.621874+0000","last_peered":"2024-04-15T15:25:57.621874+0000","last_clean":"2024-04-15T15:25:57.621874+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:25:57.621874+0000","last_undegraded":"2024-04-15T15:25:57.621874+0000","last_fullsized":"2024-04-15T15:25:57.621874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":755012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":165170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":170643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":164658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":247534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":247667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1028096,"data_stored":761144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:26:02.977 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-15T15:26:02.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:26:03.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.190+0000 7f13a8400640 1 Processor -- start 2024-04-15T15:26:03.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.190+0000 7f13a8400640 1 -- start start 2024-04-15T15:26:03.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.192+0000 7f13a8400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a00728f0 0x7f13a015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.192+0000 7f13a8400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a01485d0 con 0x7f13a01481f0 2024-04-15T15:26:03.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.192+0000 7f13a8400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a00aef20 con 0x7f13a00affc0 2024-04-15T15:26:03.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.192+0000 7f13a8400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13a015ffc0 con 0x7f13a00728f0 2024-04-15T15:26:03.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.192+0000 7f13a5a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f13a01481f0 0x7f13a00721e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38378/0 (socket says 172.21.15.150:38378) 2024-04-15T15:26:03.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.192+0000 7f13a5a00640 1 -- 172.21.15.150:0/1550704569 learned_addr learned my addr 172.21.15.150:0/1550704569 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:03.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.192+0000 7f13a5000640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a00728f0 0x7f13a015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829233011 0 0) 0x7f13a01485d0 con 0x7f13a01481f0 2024-04-15T15:26:03.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1388003520 con 0x7f13a01481f0 2024-04-15T15:26:03.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297113971 0 0) 0x7f13a00aef20 con 0x7f13a00affc0 2024-04-15T15:26:03.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13a01485d0 con 0x7f13a00affc0 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2542143525 0 0) 0x7f1388003520 con 0x7f13a01481f0 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a00728f0 msgr2=0x7f13a015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a00728f0 0x7f13a015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 >> v1:172.21.15.187:6789/0 conn(0x7f13a00affc0 legacy=0x7f13a01601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a00af200 con 0x7f13a01481f0 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f1394001670 con 0x7f13a01481f0 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 >> v1:172.21.15.150:6789/0 conn(0x7f13a01481f0 legacy=0x7f13a00721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1388003b70 0x7f1388003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388004800 0x7f1388004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a00728f0 0x7f13880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f138800ba70 con 0x7f1388003b70 2024-04-15T15:26:03.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f138800bbf0 con 0x7f1388004800 2024-04-15T15:26:03.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.193+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f138800bdf0 con 0x7f13a00728f0 2024-04-15T15:26:03.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a5a00640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1388003b70 0x7f1388003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a5000640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388004800 0x7f1388004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a6400640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a00728f0 0x7f13880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a5a00640 1 -- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a00728f0 msgr2=0x7f13880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:03.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a5a00640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a00728f0 0x7f13880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a5a00640 1 -- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388004800 msgr2=0x7f1388004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:03.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a5a00640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388004800 0x7f1388004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a5a00640 1 -- 172.21.15.150:0/1550704569 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1388003520 con 0x7f1388003b70 2024-04-15T15:26:03.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.194+0000 7f13a5000640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388004800 0x7f1388004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:26:03.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.195+0000 7f13a5a00640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1388003b70 0x7f1388003f70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1394001eb0 tx=0x7f1388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fa00ec4ac060b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:03.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.195+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394007ce0 con 0x7f1388003b70 2024-04-15T15:26:03.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.195+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1394030070 con 0x7f1388003b70 2024-04-15T15:26:03.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.195+0000 7f139fe00640 1 -- 172.21.15.150:0/1550704569 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394005d10 con 0x7f1388003b70 2024-04-15T15:26:03.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.196+0000 7f13a8400640 1 -- 172.21.15.150:0/1550704569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1388003b70 msgr2=0x7f1388003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.196+0000 7f13a8400640 1 --2- 172.21.15.150:0/1550704569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1388003b70 0x7f1388003f70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1394001eb0 tx=0x7f1388005830 comp rx=0 tx=0).stop 2024-04-15T15:26:03.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.196+0000 7f13a6400640 1 -- 172.21.15.150:0/1550704569 reap_dead start 2024-04-15T15:26:03.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.196+0000 7f13a8400640 1 -- 172.21.15.150:0/1550704569 shutdown_connections 2024-04-15T15:26:03.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.196+0000 7f13a8400640 1 -- 172.21.15.150:0/1550704569 >> 172.21.15.150:0/1550704569 conn(0x7f13a00b27b0 msgr2=0x7f13a00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:03.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.196+0000 7f13a8400640 1 -- 172.21.15.150:0/1550704569 shutdown_connections 2024-04-15T15:26:03.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.196+0000 7f13a8400640 1 -- 172.21.15.150:0/1550704569 wait complete. 2024-04-15T15:26:03.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.198+0000 7f13a8400640 1 Processor -- start 2024-04-15T15:26:03.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.198+0000 7f13a8400640 1 -- start start 2024-04-15T15:26:03.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a8400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388003b70 0x7f13a01f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a8400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13a00ce6f0 0x7f13a01606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a8400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a015e420 0x7f13a0160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a8400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00afaa0 con 0x7f13a00ce6f0 2024-04-15T15:26:03.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a8400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00af7c0 con 0x7f1388003b70 2024-04-15T15:26:03.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a8400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13a00b5560 con 0x7f13a015e420 2024-04-15T15:26:03.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a5000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13a00ce6f0 0x7f13a01606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a5000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13a00ce6f0 0x7f13a01606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51034/0 (socket says 172.21.15.150:51034) 2024-04-15T15:26:03.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.199+0000 7f13a5000640 1 -- 172.21.15.150:0/4198165612 learned_addr learned my addr 172.21.15.150:0/4198165612 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:03.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a5a00640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388003b70 0x7f13a01f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a6400640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a015e420 0x7f13a0160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a5000640 1 -- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a015e420 msgr2=0x7f13a0160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a5000640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a015e420 0x7f13a0160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a5000640 1 -- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388003b70 msgr2=0x7f13a01f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a5000640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388003b70 0x7f13a01f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a5000640 1 -- 172.21.15.150:0/4198165612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01f5af0 con 0x7f13a00ce6f0 2024-04-15T15:26:03.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a5a00640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388003b70 0x7f13a01f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:03.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a6400640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a015e420 0x7f13a0160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:03.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a5000640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13a00ce6f0 0x7f13a01606a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f138c011700 tx=0x7f138c011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:03.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f139d600640 1 -- 172.21.15.150:0/4198165612 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c00f9c0 con 0x7f13a00ce6f0 2024-04-15T15:26:03.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f139d600640 1 -- 172.21.15.150:0/4198165612 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138c00fb60 con 0x7f13a00ce6f0 2024-04-15T15:26:03.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f139d600640 1 -- 172.21.15.150:0/4198165612 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c015460 con 0x7f13a00ce6f0 2024-04-15T15:26:03.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.200+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a00cd650 con 0x7f13a00ce6f0 2024-04-15T15:26:03.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.201+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a00cdc10 con 0x7f13a00ce6f0 2024-04-15T15:26:03.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.202+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a0068c80 con 0x7f13a00ce6f0 2024-04-15T15:26:03.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.203+0000 7f139d600640 1 -- 172.21.15.150:0/4198165612 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f138c015600 con 0x7f13a00ce6f0 2024-04-15T15:26:03.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.204+0000 7f139d600640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f138006c9b0 0x7f138006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.204+0000 7f139d600640 1 -- 172.21.15.150:0/4198165612 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f138c08c390 con 0x7f13a00ce6f0 2024-04-15T15:26:03.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.205+0000 7f13a5a00640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f138006c9b0 0x7f138006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.206+0000 7f13a5a00640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f138006c9b0 0x7f138006ee70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1394001eb0 tx=0x7f1394007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:03.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.209+0000 7f139d600640 1 -- 172.21.15.150:0/4198165612 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13a01f5af0 con 0x7f13a00ce6f0 2024-04-15T15:26:03.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.404+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13a00b2de0 con 0x7f138006c9b0 2024-04-15T15:26:03.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.406+0000 7f139d600640 1 -- 172.21.15.150:0/4198165612 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13496 (secure 0 0 0) 0x7f13940484d0 con 0x7f138006c9b0 2024-04-15T15:26:03.408 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:26:03.409 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:26:03.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f138006c9b0 msgr2=0x7f138006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f138006c9b0 0x7f138006ee70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1394001eb0 tx=0x7f1394007000 comp rx=0 tx=0).stop 2024-04-15T15:26:03.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13a00ce6f0 msgr2=0x7f13a01606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13a00ce6f0 0x7f13a01606a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f138c011700 tx=0x7f138c011bc0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 shutdown_connections 2024-04-15T15:26:03.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f138006c9b0 0x7f138006ee70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13a015e420 0x7f13a0160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13a00ce6f0 0x7f13a01606a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 --2- 172.21.15.150:0/4198165612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1388003b70 0x7f13a01f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 >> 172.21.15.150:0/4198165612 conn(0x7f13a00b27b0 msgr2=0x7f13a0146b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:03.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 shutdown_connections 2024-04-15T15:26:03.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.409+0000 7f13a8400640 1 -- 172.21.15.150:0/4198165612 wait complete. 2024-04-15T15:26:03.430 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-15T15:26:03.261984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":225000,"kb_used_data":4676,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749720,"statfs":{"total":773094113280,"available":772863713280,"internally_reserved":0,"allocated":4788224,"data_stored":2688122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-74746,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-132,"num_write":-106,"num_write_kb":-118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.835057"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-15T15:25:57.621874+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:25:57.621874+0000","last_peered":"2024-04-15T15:25:57.621874+0000","last_clean":"2024-04-15T15:25:57.621874+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:25:57.621874+0000","last_undegraded":"2024-04-15T15:25:57.621874+0000","last_fullsized":"2024-04-15T15:25:57.621874+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":755012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":165170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":170643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":164658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":247534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":247667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1028096,"data_stored":761144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:26:03.430 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-15T15:26:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.644+0000 7f607f200640 1 Processor -- start 2024-04-15T15:26:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.644+0000 7f607f200640 1 -- start start 2024-04-15T15:26:03.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.646+0000 7f607f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f608015d2f0 0x7f608006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.646+0000 7f607f200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f608015dbb0 con 0x7f6080072660 2024-04-15T15:26:03.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.646+0000 7f607f200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f608015d8d0 con 0x7f60800cee00 2024-04-15T15:26:03.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.646+0000 7f607f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f608006f4f0 con 0x7f608015d2f0 2024-04-15T15:26:03.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.646+0000 7f6077e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6080072660 0x7f608015cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38392/0 (socket says 172.21.15.150:38392) 2024-04-15T15:26:03.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.646+0000 7f6077e00640 1 -- 172.21.15.150:0/632816496 learned_addr learned my addr 172.21.15.150:0/632816496 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.646+0000 7f607e800640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f608015d2f0 0x7f608006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.646+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427622626 0 0) 0x7f608015dbb0 con 0x7f6080072660 2024-04-15T15:26:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6064003520 con 0x7f6080072660 2024-04-15T15:26:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 117758967 0 0) 0x7f6064003520 con 0x7f6080072660 2024-04-15T15:26:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f608015d2f0 msgr2=0x7f608006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f608015d2f0 0x7f608006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 >> v1:172.21.15.187:6789/0 conn(0x7f60800cee00 legacy=0x7f6080071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608006dde0 con 0x7f6080072660 2024-04-15T15:26:03.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f6068001670 con 0x7f6080072660 2024-04-15T15:26:03.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 >> v1:172.21.15.150:6789/0 conn(0x7f6080072660 legacy=0x7f608015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f608015d2f0 0x7f60640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004010 0x7f6064004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6064004d40 0x7f60640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f606400bb50 con 0x7f6064004d40 2024-04-15T15:26:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f606400bd00 con 0x7f6064004010 2024-04-15T15:26:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f606400bf00 con 0x7f608015d2f0 2024-04-15T15:26:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.647+0000 7f607e800640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6064004d40 0x7f60640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.648+0000 7f6077e00640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004010 0x7f6064004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.648+0000 7f607de00640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f608015d2f0 0x7f60640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.648+0000 7f607e800640 1 -- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f608015d2f0 msgr2=0x7f60640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.648+0000 7f607e800640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f608015d2f0 0x7f60640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.648+0000 7f607e800640 1 -- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004010 msgr2=0x7f6064004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:03.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.648+0000 7f607e800640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004010 0x7f6064004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.648+0000 7f607e800640 1 -- 172.21.15.150:0/632816496 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6064003520 con 0x7f6064004d40 2024-04-15T15:26:03.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.649+0000 7f607e800640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6064004d40 0x7f60640051c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6078007360 tx=0x7f6064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2d3d449f6ec806 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:03.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.649+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078039070 con 0x7f6064004d40 2024-04-15T15:26:03.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.649+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607802ccd0 con 0x7f6064004d40 2024-04-15T15:26:03.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.649+0000 7f607d400640 1 -- 172.21.15.150:0/632816496 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078009be0 con 0x7f6064004d40 2024-04-15T15:26:03.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.650+0000 7f607f200640 1 -- 172.21.15.150:0/632816496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6064004d40 msgr2=0x7f60640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.650+0000 7f607f200640 1 --2- 172.21.15.150:0/632816496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6064004d40 0x7f60640051c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6078007360 tx=0x7f6064005830 comp rx=0 tx=0).stop 2024-04-15T15:26:03.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.650+0000 7f607e800640 1 -- 172.21.15.150:0/632816496 reap_dead start 2024-04-15T15:26:03.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.650+0000 7f607f200640 1 -- 172.21.15.150:0/632816496 shutdown_connections 2024-04-15T15:26:03.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.650+0000 7f607f200640 1 -- 172.21.15.150:0/632816496 >> 172.21.15.150:0/632816496 conn(0x7f608006b930 msgr2=0x7f608006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:03.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.650+0000 7f607f200640 1 -- 172.21.15.150:0/632816496 shutdown_connections 2024-04-15T15:26:03.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.650+0000 7f607f200640 1 -- 172.21.15.150:0/632816496 wait complete. 2024-04-15T15:26:03.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.652+0000 7f607f200640 1 Processor -- start 2024-04-15T15:26:03.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.652+0000 7f607f200640 1 -- start start 2024-04-15T15:26:03.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004d40 0x7f60800abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6080151f20 0x7f6080154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60801552c0 0x7f608015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f608006e8c0 con 0x7f60801552c0 2024-04-15T15:26:03.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f608006e3a0 con 0x7f6064004d40 2024-04-15T15:26:03.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f608006ea40 con 0x7f6080151f20 2024-04-15T15:26:03.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60801552c0 0x7f608015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60801552c0 0x7f608015dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51060/0 (socket says 172.21.15.150:51060) 2024-04-15T15:26:03.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607e800640 1 -- 172.21.15.150:0/3226227878 learned_addr learned my addr 172.21.15.150:0/3226227878 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:03.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f607de00640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004d40 0x7f60800abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.653+0000 7f6077e00640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6080151f20 0x7f6080154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f607e800640 1 -- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6080151f20 msgr2=0x7f6080154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f607e800640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6080151f20 0x7f6080154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f607e800640 1 -- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004d40 msgr2=0x7f60800abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f607e800640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004d40 0x7f60800abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f607e800640 1 -- 172.21.15.150:0/3226227878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60800ac3d0 con 0x7f60801552c0 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f607e800640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60801552c0 0x7f608015dd90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6078007480 tx=0x7f6078004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f6076000640 1 -- 172.21.15.150:0/3226227878 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078039070 con 0x7f60801552c0 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f6076000640 1 -- 172.21.15.150:0/3226227878 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607803a040 con 0x7f60801552c0 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608015eae0 con 0x7f60801552c0 2024-04-15T15:26:03.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f6076000640 1 -- 172.21.15.150:0/3226227878 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078009be0 con 0x7f60801552c0 2024-04-15T15:26:03.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.654+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608015fa00 con 0x7f60801552c0 2024-04-15T15:26:03.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.656+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60800755c0 con 0x7f60801552c0 2024-04-15T15:26:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.657+0000 7f6076000640 1 -- 172.21.15.150:0/3226227878 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6078046030 con 0x7f60801552c0 2024-04-15T15:26:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.657+0000 7f6076000640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f605c06c9b0 0x7f605c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.658+0000 7f6076000640 1 -- 172.21.15.150:0/3226227878 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f60780ad190 con 0x7f60801552c0 2024-04-15T15:26:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.658+0000 7f607de00640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f605c06c9b0 0x7f605c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.659+0000 7f607de00640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f605c06c9b0 0x7f605c06ee70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f608015b910 tx=0x7f6070009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.662+0000 7f6076000640 1 -- 172.21.15.150:0/3226227878 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f607804d060 con 0x7f60801552c0 2024-04-15T15:26:03.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.862+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f608015aa80 con 0x7f60801552c0 2024-04-15T15:26:03.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.864+0000 7f6076000640 1 -- 172.21.15.150:0/3226227878 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13202 (secure 0 0 0) 0x7f607807b800 con 0x7f60801552c0 2024-04-15T15:26:03.866 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:26:03.866 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":39,"fsid":"77b40cf9-eeeb-4ecf-83de-8ece60bb9ce4","created":"2024-04-15T15:23:30.097101+0000","modified":"2024-04-15T15:25:58.428891+0000","last_up_change":"2024-04-15T15:23:43.103847+0000","last_in_change":"2024-04-15T15:23:34.049102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-15T15:23:43.252042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9eade02f-52ed-4fe2-9498-a568d26e296e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6825","nonce":2496520552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6827","nonce":2496520552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6831","nonce":2496520552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2496520552},{"type":"v1","addr":"172.21.15.150:6829","nonce":2496520552}]},"public_addr":"172.21.15.150:6825/2496520552","cluster_addr":"172.21.15.150:6827/2496520552","heartbeat_back_addr":"172.21.15.150:6831/2496520552","heartbeat_front_addr":"172.21.15.150:6829/2496520552","state":["exists","up"]},{"osd":1,"uuid":"7dd7200c-18bc-4ded-9d75-f9f7be236c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6801","nonce":2508444218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6803","nonce":2508444218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6807","nonce":2508444218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2508444218},{"type":"v1","addr":"172.21.15.150:6805","nonce":2508444218}]},"public_addr":"172.21.15.150:6801/2508444218","cluster_addr":"172.21.15.150:6803/2508444218","heartbeat_back_addr":"172.21.15.150:6807/2508444218","heartbeat_front_addr":"172.21.15.150:6805/2508444218","state":["exists","up"]},{"osd":2,"uuid":"ddd19ed9-014c-4e0a-928c-21833f47ae17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6809","nonce":1792706494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6811","nonce":1792706494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6815","nonce":1792706494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1792706494},{"type":"v1","addr":"172.21.15.150:6813","nonce":1792706494}]},"public_addr":"172.21.15.150:6809/1792706494","cluster_addr":"172.21.15.150:6811/1792706494","heartbeat_back_addr":"172.21.15.150:6815/1792706494","heartbeat_front_addr":"172.21.15.150:6813/1792706494","state":["exists","up"]},{"osd":3,"uuid":"fd9d396e-995d-4e3b-87b2-677311a2d3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6817","nonce":3354529328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6819","nonce":3354529328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6823","nonce":3354529328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3354529328},{"type":"v1","addr":"172.21.15.150:6821","nonce":3354529328}]},"public_addr":"172.21.15.150:6817/3354529328","cluster_addr":"172.21.15.150:6819/3354529328","heartbeat_back_addr":"172.21.15.150:6823/3354529328","heartbeat_front_addr":"172.21.15.150:6821/3354529328","state":["exists","up"]},{"osd":4,"uuid":"03203baa-ce38-45e6-b112-6b81c463ac07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6812","nonce":4034856676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6818","nonce":4034856676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6827","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6830","nonce":4034856676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4034856676},{"type":"v1","addr":"172.21.15.187:6823","nonce":4034856676}]},"public_addr":"172.21.15.187:6812/4034856676","cluster_addr":"172.21.15.187:6818/4034856676","heartbeat_back_addr":"172.21.15.187:6830/4034856676","heartbeat_front_addr":"172.21.15.187:6823/4034856676","state":["exists","up"]},{"osd":5,"uuid":"969f8b07-ebef-468f-b483-2423dedd8bdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6801","nonce":2717020412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6803","nonce":2717020412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6807","nonce":2717020412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2717020412},{"type":"v1","addr":"172.21.15.187:6805","nonce":2717020412}]},"public_addr":"172.21.15.187:6801/2717020412","cluster_addr":"172.21.15.187:6803/2717020412","heartbeat_back_addr":"172.21.15.187:6807/2717020412","heartbeat_front_addr":"172.21.15.187:6805/2717020412","state":["exists","up"]},{"osd":6,"uuid":"a5f188e2-5dea-4b4a-b561-f4686758e7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6814","nonce":455912281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6821","nonce":455912281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6829","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6831","nonce":455912281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":455912281},{"type":"v1","addr":"172.21.15.187:6826","nonce":455912281}]},"public_addr":"172.21.15.187:6814/455912281","cluster_addr":"172.21.15.187:6821/455912281","heartbeat_back_addr":"172.21.15.187:6831/455912281","heartbeat_front_addr":"172.21.15.187:6826/455912281","state":["exists","up"]},{"osd":7,"uuid":"89666b85-6bae-44f7-b9b1-4a69c9f8e436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6810","nonce":2186687773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6816","nonce":2186687773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6828","nonce":2186687773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6819","nonce":2186687773},{"type":"v1","addr":"172.21.15.187:6822","nonce":2186687773}]},"public_addr":"172.21.15.187:6810/2186687773","cluster_addr":"172.21.15.187:6816/2186687773","heartbeat_back_addr":"172.21.15.187:6828/2186687773","heartbeat_front_addr":"172.21.15.187:6822/2186687773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.431717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.369135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.179420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.399340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.415866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.372513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.497000+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-15T15:23:39.445412+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/3243584315":"2024-04-16T15:25:55.417414+0000","172.21.15.150:6833/3243584315":"2024-04-16T15:25:55.417414+0000","172.21.15.150:6835/3327940880":"2024-04-16T15:25:33.990360+0000","172.21.15.150:6833/3327940880":"2024-04-16T15:25:33.990360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-15T15:26:03.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f605c06c9b0 msgr2=0x7f605c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f605c06c9b0 0x7f605c06ee70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f608015b910 tx=0x7f6070009040 comp rx=0 tx=0).stop 2024-04-15T15:26:03.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60801552c0 msgr2=0x7f608015dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:03.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60801552c0 0x7f608015dd90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6078007480 tx=0x7f6078004ab0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 shutdown_connections 2024-04-15T15:26:03.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60801552c0 0x7f608015dd90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f605c06c9b0 0x7f605c06ee70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6080151f20 0x7f6080154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 --2- 172.21.15.150:0/3226227878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6064004d40 0x7f60800abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:03.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 >> 172.21.15.150:0/3226227878 conn(0x7f608006b930 msgr2=0x7f6080157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:03.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 shutdown_connections 2024-04-15T15:26:03.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:03.867+0000 7f607f200640 1 -- 172.21.15.150:0/3226227878 wait complete. 2024-04-15T15:26:04.889 INFO:tasks.ceph:Scrubbing osd.0 2024-04-15T15:26:04.889 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:26:05.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.113+0000 7f52afe00640 1 Processor -- start 2024-04-15T15:26:05.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.114+0000 7f52afe00640 1 -- start start 2024-04-15T15:26:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.115+0000 7f52afe00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52b00cec60 0x7f52b0071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.115+0000 7f52afe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b0072450 0x7f52b015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.115+0000 7f52afe00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b013fc70 con 0x7f52b015d250 2024-04-15T15:26:05.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.115+0000 7f52afe00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f52b013f9e0 con 0x7f52b00cec60 2024-04-15T15:26:05.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.115+0000 7f52afe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52b0072890 con 0x7f52b0072450 2024-04-15T15:26:05.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52aea00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52b00cec60 0x7f52b0071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52af400640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f52b015d250 0x7f52b006f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38396/0 (socket says 172.21.15.150:38396) 2024-04-15T15:26:05.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52af400640 1 -- 172.21.15.150:0/2254178629 learned_addr learned my addr 172.21.15.150:0/2254178629 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:05.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52aea00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52b00cec60 0x7f52b0071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51062/0 (socket says 172.21.15.150:51062) 2024-04-15T15:26:05.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52a6000640 1 --2- 172.21.15.150:0/2254178629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b0072450 0x7f52b015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52aea00640 1 -- 172.21.15.150:0/2254178629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b0072450 msgr2=0x7f52b015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:05.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52aea00640 1 --2- 172.21.15.150:0/2254178629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b0072450 0x7f52b015cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52aea00640 1 -- 172.21.15.150:0/2254178629 >> v1:172.21.15.150:6789/0 conn(0x7f52b015d250 legacy=0x7f52b006f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52aea00640 1 -- 172.21.15.150:0/2254178629 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b0141a00 con 0x7f52b00cec60 2024-04-15T15:26:05.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.116+0000 7f52a6000640 1 --2- 172.21.15.150:0/2254178629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b0072450 0x7f52b015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.117+0000 7f52aea00640 1 --2- 172.21.15.150:0/2254178629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b00cec60 0x7f52b0071f10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b78ad52b193f0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.117+0000 7f52ae000640 1 -- 172.21.15.150:0/2254178629 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0034070 con 0x7f52b00cec60 2024-04-15T15:26:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52ae000640 1 -- 172.21.15.150:0/2254178629 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a0030800 con 0x7f52b00cec60 2024-04-15T15:26:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52ae000640 1 -- 172.21.15.150:0/2254178629 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0030bd0 con 0x7f52b00cec60 2024-04-15T15:26:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52afe00640 1 -- 172.21.15.150:0/2254178629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b00cec60 msgr2=0x7f52b0071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52afe00640 1 --2- 172.21.15.150:0/2254178629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b00cec60 0x7f52b0071f10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a002b940 comp rx=0 tx=0).stop 2024-04-15T15:26:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52afe00640 1 -- 172.21.15.150:0/2254178629 shutdown_connections 2024-04-15T15:26:05.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52afe00640 1 --2- 172.21.15.150:0/2254178629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b0072450 0x7f52b015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52afe00640 1 --2- 172.21.15.150:0/2254178629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b00cec60 0x7f52b0071f10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52afe00640 1 -- 172.21.15.150:0/2254178629 >> 172.21.15.150:0/2254178629 conn(0x7f52b00b6170 msgr2=0x7f52b006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52afe00640 1 -- 172.21.15.150:0/2254178629 shutdown_connections 2024-04-15T15:26:05.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.118+0000 7f52afe00640 1 -- 172.21.15.150:0/2254178629 wait complete. 2024-04-15T15:26:05.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.120+0000 7f52afe00640 1 Processor -- start 2024-04-15T15:26:05.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.120+0000 7f52afe00640 1 -- start start 2024-04-15T15:26:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.121+0000 7f52afe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0072450 0x7f52b012f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.121+0000 7f52afe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b00cec60 0x7f52b0159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.121+0000 7f52afe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b015d250 0x7f52b012d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.121+0000 7f52afe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01401f0 con 0x7f52b015d250 2024-04-15T15:26:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.121+0000 7f52afe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52b013ff10 con 0x7f52b0072450 2024-04-15T15:26:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.121+0000 7f52afe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52b01404d0 con 0x7f52b00cec60 2024-04-15T15:26:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.121+0000 7f52af400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b015d250 0x7f52b012d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52af400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b015d250 0x7f52b012d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51072/0 (socket says 172.21.15.150:51072) 2024-04-15T15:26:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52af400640 1 -- 172.21.15.150:0/2016583204 learned_addr learned my addr 172.21.15.150:0/2016583204 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52aea00640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0072450 0x7f52b012f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52a6000640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b00cec60 0x7f52b0159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52af400640 1 -- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b00cec60 msgr2=0x7f52b0159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52af400640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b00cec60 0x7f52b0159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52af400640 1 -- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0072450 msgr2=0x7f52b012f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52af400640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0072450 0x7f52b012f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52af400640 1 -- 172.21.15.150:0/2016583204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b012e010 con 0x7f52b015d250 2024-04-15T15:26:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52a6000640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b00cec60 0x7f52b0159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.122+0000 7f52af400640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b015d250 0x7f52b012d560 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f52a800ca80 tx=0x7f52a800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.123+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8017070 con 0x7f52b015d250 2024-04-15T15:26:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.123+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a800ac30 con 0x7f52b015d250 2024-04-15T15:26:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.123+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8012600 con 0x7f52b015d250 2024-04-15T15:26:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.123+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b012e2f0 con 0x7f52b015d250 2024-04-15T15:26:05.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.123+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b012eb90 con 0x7f52b015d250 2024-04-15T15:26:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.124+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52b0068b40 con 0x7f52b015d250 2024-04-15T15:26:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.125+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f52a801a070 con 0x7f52b015d250 2024-04-15T15:26:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.126+0000 7f52a7e00640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f528c06c980 0x7f528c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.126+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f52a808b570 con 0x7f52b015d250 2024-04-15T15:26:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.126+0000 7f52a7e00640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7f528c0728e0 0x7f528c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.126+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 --> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52a8020e50 con 0x7f528c0728e0 2024-04-15T15:26:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.126+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f52a8092050 con 0x7f52b015d250 2024-04-15T15:26:05.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.126+0000 7f52a6000640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7f528c0728e0 0x7f528c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.127+0000 7f52aea00640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f528c06c980 0x7f528c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.127+0000 7f52a6000640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7f528c0728e0 0x7f528c074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.128+0000 7f52aea00640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f528c06c980 0x7f528c06ee40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f52a0036000 tx=0x7f52a003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.128+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 <== osd.0 v2:172.21.15.150:6824/2496520552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52a8020e50 con 0x7f528c0728e0 2024-04-15T15:26:05.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.142+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 --> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f52b015fdf0 con 0x7f528c0728e0 2024-04-15T15:26:05.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52a7e00640 1 -- 172.21.15.150:0/2016583204 <== osd.0 v2:172.21.15.150:6824/2496520552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f52b015fdf0 con 0x7f528c0728e0 2024-04-15T15:26:05.144 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:26:05.145 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:26:05.145 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:26:05.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7f528c0728e0 msgr2=0x7f528c074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52afe00640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.150:6824/2496520552,v1:172.21.15.150:6825/2496520552] conn(0x7f528c0728e0 0x7f528c074d00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f528c06c980 msgr2=0x7f528c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52afe00640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f528c06c980 0x7f528c06ee40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f52a0036000 tx=0x7f52a003b040 comp rx=0 tx=0).stop 2024-04-15T15:26:05.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b015d250 msgr2=0x7f52b012d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52afe00640 1 --2- 172.21.15.150:0/2016583204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b015d250 0x7f52b012d560 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f52a800ca80 tx=0x7f52a800cf40 comp rx=0 tx=0).stop 2024-04-15T15:26:05.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52af400640 1 -- 172.21.15.150:0/2016583204 reap_dead start 2024-04-15T15:26:05.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 shutdown_connections 2024-04-15T15:26:05.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.143+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 >> 172.21.15.150:0/2016583204 conn(0x7f52b00b6170 msgr2=0x7f52b006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:05.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.144+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 shutdown_connections 2024-04-15T15:26:05.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.144+0000 7f52afe00640 1 -- 172.21.15.150:0/2016583204 wait complete. 2024-04-15T15:26:05.164 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-15T15:26:05.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.377+0000 7f8d4e800640 1 Processor -- start 2024-04-15T15:26:05.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.378+0000 7f8d4e800640 1 -- start start 2024-04-15T15:26:05.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.379+0000 7f8d4e800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d480ae790 0x7f8d48162420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.379+0000 7f8d4e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d48162960 0x7f8d480b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.379+0000 7f8d4e800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d480b3e90 con 0x7f8d48148050 2024-04-15T15:26:05.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.379+0000 7f8d4e800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8d480afba0 con 0x7f8d480ae790 2024-04-15T15:26:05.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.379+0000 7f8d4e800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d480aed00 con 0x7f8d48162960 2024-04-15T15:26:05.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.380+0000 7f8d4ca00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d480ae790 0x7f8d48162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.380+0000 7f8d4de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d48162960 0x7f8d480b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.380+0000 7f8d4de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d48162960 0x7f8d480b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:52250/0 (socket says 172.21.15.150:52250) 2024-04-15T15:26:05.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.380+0000 7f8d4d400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8d48148050 0x7f8d480b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.150:52160/0 (socket says 172.21.15.150:52160) 2024-04-15T15:26:05.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.380+0000 7f8d4de00640 1 -- 172.21.15.150:0/113013207 learned_addr learned my addr 172.21.15.150:0/113013207 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.380+0000 7f8d4ca00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d480ae790 0x7f8d48162420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:35144/0 (socket says 172.21.15.150:35144) 2024-04-15T15:26:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.381+0000 7f8d4de00640 1 -- 172.21.15.150:0/113013207 >> v2:172.21.15.187:3300/0 conn(0x7f8d480ae790 msgr2=0x7f8d48162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.381+0000 7f8d4de00640 1 --2- 172.21.15.150:0/113013207 >> v2:172.21.15.187:3300/0 conn(0x7f8d480ae790 0x7f8d48162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.381+0000 7f8d4de00640 1 -- 172.21.15.150:0/113013207 >> v1:172.21.15.187:6789/0 conn(0x7f8d48148050 legacy=0x7f8d480b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.381+0000 7f8d4de00640 1 -- 172.21.15.150:0/113013207 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d480aefe0 con 0x7f8d48162960 2024-04-15T15:26:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.381+0000 7f8d4ca00640 1 --2- 172.21.15.150:0/113013207 >> v2:172.21.15.187:3300/0 conn(0x7f8d480ae790 0x7f8d48162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:26:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.382+0000 7f8d4de00640 1 --2- 172.21.15.150:0/113013207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d48162960 0x7f8d480b0360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009e50 tx=0x7f8d4002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba723d4fda687590 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.383+0000 7f8d47e00640 1 -- 172.21.15.150:0/113013207 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4002cb00 con 0x7f8d48162960 2024-04-15T15:26:05.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.383+0000 7f8d47e00640 1 -- 172.21.15.150:0/113013207 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d4002cca0 con 0x7f8d48162960 2024-04-15T15:26:05.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.383+0000 7f8d47e00640 1 -- 172.21.15.150:0/113013207 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40034610 con 0x7f8d48162960 2024-04-15T15:26:05.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.384+0000 7f8d4e800640 1 -- 172.21.15.150:0/113013207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d48162960 msgr2=0x7f8d480b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.384+0000 7f8d4e800640 1 --2- 172.21.15.150:0/113013207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d48162960 0x7f8d480b0360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009e50 tx=0x7f8d4002bf60 comp rx=0 tx=0).stop 2024-04-15T15:26:05.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.384+0000 7f8d4e800640 1 -- 172.21.15.150:0/113013207 shutdown_connections 2024-04-15T15:26:05.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.384+0000 7f8d4e800640 1 --2- 172.21.15.150:0/113013207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d48162960 0x7f8d480b0360 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.384+0000 7f8d4e800640 1 --2- 172.21.15.150:0/113013207 >> v2:172.21.15.187:3300/0 conn(0x7f8d480ae790 0x7f8d48162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.384+0000 7f8d4e800640 1 -- 172.21.15.150:0/113013207 >> 172.21.15.150:0/113013207 conn(0x7f8d480b23a0 msgr2=0x7f8d480b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:05.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.384+0000 7f8d4e800640 1 -- 172.21.15.150:0/113013207 shutdown_connections 2024-04-15T15:26:05.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.384+0000 7f8d4e800640 1 -- 172.21.15.150:0/113013207 wait complete. 2024-04-15T15:26:05.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.386+0000 7f8d4e800640 1 Processor -- start 2024-04-15T15:26:05.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.386+0000 7f8d4e800640 1 -- start start 2024-04-15T15:26:05.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.387+0000 7f8d4e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ae790 0x7f8d481433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.387+0000 7f8d4e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d48148050 0x7f8d4815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.387+0000 7f8d4e800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48162960 0x7f8d4815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.387+0000 7f8d4e800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8d48148430 con 0x7f8d48148050 2024-04-15T15:26:05.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.387+0000 7f8d4e800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d480afe40 con 0x7f8d48162960 2024-04-15T15:26:05.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.387+0000 7f8d4e800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d481485b0 con 0x7f8d480ae790 2024-04-15T15:26:05.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d48148050 0x7f8d4815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4ca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d48148050 0x7f8d4815e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51084/0 (socket says 172.21.15.150:51084) 2024-04-15T15:26:05.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4ca00640 1 -- 172.21.15.150:0/234192705 learned_addr learned my addr 172.21.15.150:0/234192705 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:05.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4d400640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ae790 0x7f8d481433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4de00640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48162960 0x7f8d4815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4ca00640 1 -- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ae790 msgr2=0x7f8d481433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4ca00640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ae790 0x7f8d481433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4ca00640 1 -- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48162960 msgr2=0x7f8d4815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4ca00640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48162960 0x7f8d4815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.388+0000 7f8d4ca00640 1 -- 172.21.15.150:0/234192705 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4815f640 con 0x7f8d48148050 2024-04-15T15:26:05.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.389+0000 7f8d4ca00640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d48148050 0x7f8d4815e390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400ebe0 tx=0x7f8d3400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.389+0000 7f8d45600640 1 -- 172.21.15.150:0/234192705 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34017070 con 0x7f8d48148050 2024-04-15T15:26:05.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.389+0000 7f8d45600640 1 -- 172.21.15.150:0/234192705 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3400cc30 con 0x7f8d48148050 2024-04-15T15:26:05.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.389+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4815f920 con 0x7f8d48148050 2024-04-15T15:26:05.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.389+0000 7f8d45600640 1 -- 172.21.15.150:0/234192705 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34013470 con 0x7f8d48148050 2024-04-15T15:26:05.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.389+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4815fee0 con 0x7f8d48148050 2024-04-15T15:26:05.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.390+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d480b3240 con 0x7f8d48148050 2024-04-15T15:26:05.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.391+0000 7f8d45600640 1 -- 172.21.15.150:0/234192705 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8d3401a070 con 0x7f8d48148050 2024-04-15T15:26:05.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.392+0000 7f8d45600640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d2806c980 0x7f8d2806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.392+0000 7f8d45600640 1 -- 172.21.15.150:0/234192705 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f8d3408b990 con 0x7f8d48148050 2024-04-15T15:26:05.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.393+0000 7f8d4d400640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d2806c980 0x7f8d2806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.394+0000 7f8d4d400640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d2806c980 0x7f8d2806ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d48072110 tx=0x7f8d3c007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.398+0000 7f8d45600640 1 -- 172.21.15.150:0/234192705 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d34059f70 con 0x7f8d48148050 2024-04-15T15:26:05.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.600+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f8d48071e60 con 0x7f8d2806c980 2024-04-15T15:26:05.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.602+0000 7f8d45600640 1 -- 172.21.15.150:0/234192705 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8d48071e60 con 0x7f8d2806c980 2024-04-15T15:26:05.603 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 0 to deep-scrub 2024-04-15T15:26:05.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d2806c980 msgr2=0x7f8d2806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d2806c980 0x7f8d2806ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d48072110 tx=0x7f8d3c007ce0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d48148050 msgr2=0x7f8d4815e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d48148050 0x7f8d4815e390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400ebe0 tx=0x7f8d3400c520 comp rx=0 tx=0).stop 2024-04-15T15:26:05.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 shutdown_connections 2024-04-15T15:26:05.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48162960 0x7f8d4815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d48148050 0x7f8d4815e390 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f8d2806c980 0x7f8d2806ee40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 --2- 172.21.15.150:0/234192705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d480ae790 0x7f8d481433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 >> 172.21.15.150:0/234192705 conn(0x7f8d480b23a0 msgr2=0x7f8d4806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:05.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 shutdown_connections 2024-04-15T15:26:05.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.604+0000 7f8d4e800640 1 -- 172.21.15.150:0/234192705 wait complete. 2024-04-15T15:26:05.626 INFO:tasks.ceph:Scrubbing osd.1 2024-04-15T15:26:05.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:26:05.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.842+0000 7f71c3200640 1 Processor -- start 2024-04-15T15:26:05.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.842+0000 7f71c3200640 1 -- start start 2024-04-15T15:26:05.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c3200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f71c40cec60 0x7f71c4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71c4072450 0x7f71c415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c3200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c413fc70 con 0x7f71c415d250 2024-04-15T15:26:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c3200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f71c413f9e0 con 0x7f71c40cec60 2024-04-15T15:26:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f71c4072890 con 0x7f71c4072450 2024-04-15T15:26:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c2800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f71c415d250 0x7f71c406f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38400/0 (socket says 172.21.15.150:38400) 2024-04-15T15:26:05.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c2800640 1 -- 172.21.15.150:0/4225495366 learned_addr learned my addr 172.21.15.150:0/4225495366 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c1e00640 1 --2- 172.21.15.150:0/4225495366 >> v2:172.21.15.187:3300/0 conn(0x7f71c40cec60 0x7f71c4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.844+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3171485831 0 0) 0x7f71c413fc70 con 0x7f71c415d250 2024-04-15T15:26:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71a8003520 con 0x7f71c415d250 2024-04-15T15:26:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c1400640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71c4072450 0x7f71c415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3310451299 0 0) 0x7f71a8003520 con 0x7f71c415d250 2024-04-15T15:26:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71c4072450 msgr2=0x7f71c415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:05.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71c4072450 0x7f71c415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 >> v2:172.21.15.187:3300/0 conn(0x7f71c40cec60 msgr2=0x7f71c4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 --2- 172.21.15.150:0/4225495366 >> v2:172.21.15.187:3300/0 conn(0x7f71c40cec60 0x7f71c4071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c4141a00 con 0x7f71c415d250 2024-04-15T15:26:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f71bc0019f0 con 0x7f71c415d250 2024-04-15T15:26:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 >> v1:172.21.15.150:6789/0 conn(0x7f71c415d250 legacy=0x7f71c406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71a80039b0 0x7f71a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71c4072450 0x7f71a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a8004d40 0x7f71a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71c4072890 con 0x7f71a8004d40 2024-04-15T15:26:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71a800b9c0 con 0x7f71a80039b0 2024-04-15T15:26:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f71a800bbc0 con 0x7f71c4072450 2024-04-15T15:26:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.845+0000 7f71c2800640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a8004d40 0x7f71a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.846+0000 7f71c1400640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71c4072450 0x7f71a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.846+0000 7f71c1e00640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71a80039b0 0x7f71a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.846+0000 7f71c2800640 1 -- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71c4072450 msgr2=0x7f71a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:05.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.846+0000 7f71c2800640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71c4072450 0x7f71a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.846+0000 7f71c2800640 1 -- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71a80039b0 msgr2=0x7f71a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.846+0000 7f71c2800640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71a80039b0 0x7f71a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.846+0000 7f71c2800640 1 -- 172.21.15.150:0/4225495366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a8003520 con 0x7f71a8004d40 2024-04-15T15:26:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.847+0000 7f71c2800640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a8004d40 0x7f71a80051c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0013d0 tx=0x7f71a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6255197a78a4ba05 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.847+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00a410 con 0x7f71a8004d40 2024-04-15T15:26:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.847+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc034630 con 0x7f71a8004d40 2024-04-15T15:26:05.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.847+0000 7f71c0a00640 1 -- 172.21.15.150:0/4225495366 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc034a20 con 0x7f71a8004d40 2024-04-15T15:26:05.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.848+0000 7f71c3200640 1 -- 172.21.15.150:0/4225495366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a8004d40 msgr2=0x7f71a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.848+0000 7f71c3200640 1 --2- 172.21.15.150:0/4225495366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a8004d40 0x7f71a80051c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0013d0 tx=0x7f71a8005830 comp rx=0 tx=0).stop 2024-04-15T15:26:05.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.848+0000 7f71c2800640 1 -- 172.21.15.150:0/4225495366 reap_dead start 2024-04-15T15:26:05.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.848+0000 7f71c3200640 1 -- 172.21.15.150:0/4225495366 shutdown_connections 2024-04-15T15:26:05.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.848+0000 7f71c3200640 1 -- 172.21.15.150:0/4225495366 >> 172.21.15.150:0/4225495366 conn(0x7f71c40b6170 msgr2=0x7f71c406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:05.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.848+0000 7f71c3200640 1 -- 172.21.15.150:0/4225495366 shutdown_connections 2024-04-15T15:26:05.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.848+0000 7f71c3200640 1 -- 172.21.15.150:0/4225495366 wait complete. 2024-04-15T15:26:05.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.850+0000 7f71c3200640 1 Processor -- start 2024-04-15T15:26:05.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.850+0000 7f71c3200640 1 -- start start 2024-04-15T15:26:05.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.851+0000 7f71c3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71a8004d40 0x7f71c4135c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.851+0000 7f71c3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71c4154b40 0x7f71c4155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.851+0000 7f71c3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c4155ec0 0x7f71c40ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.851+0000 7f71c3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71c41401f0 con 0x7f71c4155ec0 2024-04-15T15:26:05.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.851+0000 7f71c3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71c413ff10 con 0x7f71c4154b40 2024-04-15T15:26:05.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.851+0000 7f71c3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f71c41404d0 con 0x7f71a8004d40 2024-04-15T15:26:05.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.851+0000 7f71c2800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c4155ec0 0x7f71c40ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c2800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c4155ec0 0x7f71c40ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51098/0 (socket says 172.21.15.150:51098) 2024-04-15T15:26:05.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c2800640 1 -- 172.21.15.150:0/3039134477 learned_addr learned my addr 172.21.15.150:0/3039134477 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:05.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c1e00640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71a8004d40 0x7f71c4135c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c1400640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71c4154b40 0x7f71c4155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c2800640 1 -- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71a8004d40 msgr2=0x7f71c4135c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c2800640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71a8004d40 0x7f71c4135c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c2800640 1 -- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71c4154b40 msgr2=0x7f71c4155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c2800640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71c4154b40 0x7f71c4155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c2800640 1 -- 172.21.15.150:0/3039134477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c4153d80 con 0x7f71c4155ec0 2024-04-15T15:26:05.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.852+0000 7f71c2800640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c4155ec0 0x7f71c40ae900 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0013d0 tx=0x7f71bc001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.853+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc001f70 con 0x7f71c4155ec0 2024-04-15T15:26:05.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.853+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc044070 con 0x7f71c4155ec0 2024-04-15T15:26:05.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.853+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00a3f0 con 0x7f71c4155ec0 2024-04-15T15:26:05.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.853+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c4154060 con 0x7f71c4155ec0 2024-04-15T15:26:05.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.853+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c4136160 con 0x7f71c4155ec0 2024-04-15T15:26:05.859 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.854+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71c4154720 con 0x7f71c4155ec0 2024-04-15T15:26:05.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.855+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f71bc034c10 con 0x7f71c4155ec0 2024-04-15T15:26:05.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.856+0000 7f71ba000640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71a406c980 0x7f71a406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.856+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f71bc0aec60 con 0x7f71c4155ec0 2024-04-15T15:26:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.856+0000 7f71ba000640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f71a40728e0 0x7f71a4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.856+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 --> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71bc009eb0 con 0x7f71a40728e0 2024-04-15T15:26:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.856+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71bc0af010 con 0x7f71c4155ec0 2024-04-15T15:26:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.856+0000 7f71c1400640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f71a40728e0 0x7f71a4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.857+0000 7f71c1e00640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71a406c980 0x7f71a406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.857+0000 7f71c1400640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f71a40728e0 0x7f71a4074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.858+0000 7f71c1e00640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71a406c980 0x7f71a406ee40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009a10 tx=0x7f71b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:05.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.858+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 <== osd.1 v2:172.21.15.150:6800/2508444218 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71bc009eb0 con 0x7f71a40728e0 2024-04-15T15:26:05.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.872+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 --> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f71c4068b40 con 0x7f71a40728e0 2024-04-15T15:26:05.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71ba000640 1 -- 172.21.15.150:0/3039134477 <== osd.1 v2:172.21.15.150:6800/2508444218 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f71c4068b40 con 0x7f71a40728e0 2024-04-15T15:26:05.875 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:26:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:26:05.875 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:26:05.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f71a40728e0 msgr2=0x7f71a4074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c3200640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.150:6800/2508444218,v1:172.21.15.150:6801/2508444218] conn(0x7f71a40728e0 0x7f71a4074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:05.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71a406c980 msgr2=0x7f71a406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c3200640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f71a406c980 0x7f71a406ee40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009a10 tx=0x7f71b4008040 comp rx=0 tx=0).stop 2024-04-15T15:26:05.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c4155ec0 msgr2=0x7f71c40ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:05.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c3200640 1 --2- 172.21.15.150:0/3039134477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71c4155ec0 0x7f71c40ae900 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0013d0 tx=0x7f71bc001360 comp rx=0 tx=0).stop 2024-04-15T15:26:05.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c2800640 1 -- 172.21.15.150:0/3039134477 reap_dead start 2024-04-15T15:26:05.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 shutdown_connections 2024-04-15T15:26:05.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.873+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 >> 172.21.15.150:0/3039134477 conn(0x7f71c40b6170 msgr2=0x7f71c415ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:05.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.874+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 shutdown_connections 2024-04-15T15:26:05.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:05.874+0000 7f71c3200640 1 -- 172.21.15.150:0/3039134477 wait complete. 2024-04-15T15:26:05.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-15T15:26:06.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.090+0000 7f57e7200640 1 Processor -- start 2024-04-15T15:26:06.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.091+0000 7f57e7200640 1 -- start start 2024-04-15T15:26:06.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.092+0000 7f57e7200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f57e815eab0 0x7f57e8162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.092+0000 7f57e7200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e8162b70 0x7f57e80d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.092+0000 7f57e7200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e80b78f0 con 0x7f57e80d1a90 2024-04-15T15:26:06.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.092+0000 7f57e7200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f57e80d2e50 con 0x7f57e815eab0 2024-04-15T15:26:06.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.092+0000 7f57e7200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57e8163010 con 0x7f57e8162b70 2024-04-15T15:26:06.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.092+0000 7f57e5e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f57e815eab0 0x7f57e8162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.093+0000 7f57e5e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f57e815eab0 0x7f57e8162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51106/0 (socket says 172.21.15.150:51106) 2024-04-15T15:26:06.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.093+0000 7f57e5e00640 1 -- 172.21.15.150:0/1250395034 learned_addr learned my addr 172.21.15.150:0/1250395034 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.093+0000 7f57e5400640 1 --2- 172.21.15.150:0/1250395034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e8162b70 0x7f57e80d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.093+0000 7f57e5e00640 1 -- 172.21.15.150:0/1250395034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e8162b70 msgr2=0x7f57e80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.093+0000 7f57e5e00640 1 --2- 172.21.15.150:0/1250395034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e8162b70 0x7f57e80d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.093+0000 7f57e5e00640 1 -- 172.21.15.150:0/1250395034 >> v1:172.21.15.187:6789/0 conn(0x7f57e80d1a90 legacy=0x7f57e80b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:06.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.093+0000 7f57e5e00640 1 -- 172.21.15.150:0/1250395034 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e81632f0 con 0x7f57e815eab0 2024-04-15T15:26:06.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.094+0000 7f57e5e00640 1 --2- 172.21.15.150:0/1250395034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e815eab0 0x7f57e8162630 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009870 tx=0x7f57d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537aa347e3057ae server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.094+0000 7f57e4a00640 1 -- 172.21.15.150:0/1250395034 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8035020 con 0x7f57e815eab0 2024-04-15T15:26:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e4a00640 1 -- 172.21.15.150:0/1250395034 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d8031640 con 0x7f57e815eab0 2024-04-15T15:26:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e4a00640 1 -- 172.21.15.150:0/1250395034 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d802d040 con 0x7f57e815eab0 2024-04-15T15:26:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e7200640 1 -- 172.21.15.150:0/1250395034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e815eab0 msgr2=0x7f57e8162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e7200640 1 --2- 172.21.15.150:0/1250395034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e815eab0 0x7f57e8162630 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009870 tx=0x7f57d802b8f0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e7200640 1 -- 172.21.15.150:0/1250395034 shutdown_connections 2024-04-15T15:26:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e7200640 1 --2- 172.21.15.150:0/1250395034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e8162b70 0x7f57e80d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e7200640 1 --2- 172.21.15.150:0/1250395034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e815eab0 0x7f57e8162630 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e7200640 1 -- 172.21.15.150:0/1250395034 >> 172.21.15.150:0/1250395034 conn(0x7f57e8073240 msgr2=0x7f57e8073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:06.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e7200640 1 -- 172.21.15.150:0/1250395034 shutdown_connections 2024-04-15T15:26:06.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.095+0000 7f57e7200640 1 -- 172.21.15.150:0/1250395034 wait complete. 2024-04-15T15:26:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.097+0000 7f57e7200640 1 Processor -- start 2024-04-15T15:26:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.097+0000 7f57e7200640 1 -- start start 2024-04-15T15:26:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e7200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57e80d1a90 0x7f57e814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e7200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e815eab0 0x7f57e815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e7200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e8162b70 0x7f57e8161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e7200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57e80ba590 con 0x7f57e8162b70 2024-04-15T15:26:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e7200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57e815ee90 con 0x7f57e80d1a90 2024-04-15T15:26:06.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e7200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57e80ba710 con 0x7f57e815eab0 2024-04-15T15:26:06.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e6800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e8162b70 0x7f57e8161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e6800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e8162b70 0x7f57e8161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51120/0 (socket says 172.21.15.150:51120) 2024-04-15T15:26:06.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.098+0000 7f57e6800640 1 -- 172.21.15.150:0/2057643710 learned_addr learned my addr 172.21.15.150:0/2057643710 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:06.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.099+0000 7f57e5e00640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57e80d1a90 0x7f57e814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.099+0000 7f57e5400640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e815eab0 0x7f57e815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.099+0000 7f57e6800640 1 -- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e815eab0 msgr2=0x7f57e815c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.099+0000 7f57e6800640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e815eab0 0x7f57e815c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.099+0000 7f57e6800640 1 -- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57e80d1a90 msgr2=0x7f57e814cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.099+0000 7f57e6800640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57e80d1a90 0x7f57e814cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.099+0000 7f57e6800640 1 -- 172.21.15.150:0/2057643710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e814d310 con 0x7f57e8162b70 2024-04-15T15:26:06.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.099+0000 7f57e6800640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e8162b70 0x7f57e8161020 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c780 tx=0x7f57e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.100+0000 7f57de000640 1 -- 172.21.15.150:0/2057643710 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0004b60 con 0x7f57e8162b70 2024-04-15T15:26:06.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.100+0000 7f57de000640 1 -- 172.21.15.150:0/2057643710 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e0004d00 con 0x7f57e8162b70 2024-04-15T15:26:06.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.100+0000 7f57de000640 1 -- 172.21.15.150:0/2057643710 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e000aa90 con 0x7f57e8162b70 2024-04-15T15:26:06.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.100+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e815cee0 con 0x7f57e8162b70 2024-04-15T15:26:06.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.100+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e815d4a0 con 0x7f57e8162b70 2024-04-15T15:26:06.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.101+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e80755c0 con 0x7f57e8162b70 2024-04-15T15:26:06.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.102+0000 7f57de000640 1 -- 172.21.15.150:0/2057643710 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f57e000ac30 con 0x7f57e8162b70 2024-04-15T15:26:06.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.103+0000 7f57de000640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57c406c9b0 0x7f57c406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.103+0000 7f57de000640 1 -- 172.21.15.150:0/2057643710 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f57e008bbd0 con 0x7f57e8162b70 2024-04-15T15:26:06.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.103+0000 7f57e5e00640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57c406c9b0 0x7f57c406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.105+0000 7f57e5e00640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57c406c9b0 0x7f57c406ee70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f57d8036000 tx=0x7f57d803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.107+0000 7f57de000640 1 -- 172.21.15.150:0/2057643710 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57e0020070 con 0x7f57e8162b70 2024-04-15T15:26:06.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.307+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f57e80c9650 con 0x7f57c406c9b0 2024-04-15T15:26:06.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.308+0000 7f57de000640 1 -- 172.21.15.150:0/2057643710 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f57e80c9650 con 0x7f57c406c9b0 2024-04-15T15:26:06.310 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 1 to deep-scrub 2024-04-15T15:26:06.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57c406c9b0 msgr2=0x7f57c406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57c406c9b0 0x7f57c406ee70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f57d8036000 tx=0x7f57d803c040 comp rx=0 tx=0).stop 2024-04-15T15:26:06.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e8162b70 msgr2=0x7f57e8161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e8162b70 0x7f57e8161020 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c780 tx=0x7f57e000cc40 comp rx=0 tx=0).stop 2024-04-15T15:26:06.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 shutdown_connections 2024-04-15T15:26:06.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e8162b70 0x7f57e8161020 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f57c406c9b0 0x7f57c406ee70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57e815eab0 0x7f57e815c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 --2- 172.21.15.150:0/2057643710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57e80d1a90 0x7f57e814cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 >> 172.21.15.150:0/2057643710 conn(0x7f57e8073240 msgr2=0x7f57e80d3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:06.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 shutdown_connections 2024-04-15T15:26:06.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.311+0000 7f57e7200640 1 -- 172.21.15.150:0/2057643710 wait complete. 2024-04-15T15:26:06.331 INFO:tasks.ceph:Scrubbing osd.2 2024-04-15T15:26:06.332 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:26:06.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.548+0000 7f28ff200640 1 Processor -- start 2024-04-15T15:26:06.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.548+0000 7f28ff200640 1 -- start start 2024-04-15T15:26:06.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.550+0000 7f28ff200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2900072620 0x7f290015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.550+0000 7f28ff200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f290015d250 0x7f290006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.550+0000 7f28ff200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f290013fc70 con 0x7f29000cec60 2024-04-15T15:26:06.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.550+0000 7f28ff200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f290013f9e0 con 0x7f2900072620 2024-04-15T15:26:06.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.550+0000 7f28ff200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2900141a00 con 0x7f290015d250 2024-04-15T15:26:06.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.551+0000 7f28f7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2900072620 0x7f290015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.551+0000 7f28f7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2900072620 0x7f290015cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:35198/0 (socket says 172.21.15.150:35198) 2024-04-15T15:26:06.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.551+0000 7f28f7e00640 1 -- 172.21.15.150:0/3195135801 learned_addr learned my addr 172.21.15.150:0/3195135801 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:06.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.551+0000 7f28fe800640 1 --2- 172.21.15.150:0/3195135801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f290015d250 0x7f290006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.551+0000 7f28f7e00640 1 -- 172.21.15.150:0/3195135801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f290015d250 msgr2=0x7f290006f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:06.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.551+0000 7f28f7e00640 1 --2- 172.21.15.150:0/3195135801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f290015d250 0x7f290006f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.551+0000 7f28f7e00640 1 -- 172.21.15.150:0/3195135801 >> v1:172.21.15.187:6789/0 conn(0x7f29000cec60 legacy=0x7f2900071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.551+0000 7f28f7e00640 1 -- 172.21.15.150:0/3195135801 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290015da40 con 0x7f2900072620 2024-04-15T15:26:06.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.553+0000 7f28f7e00640 1 --2- 172.21.15.150:0/3195135801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2900072620 0x7f290015cd10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f28e80098a0 tx=0x7f28e802ba00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=254c35ddc7989415 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.553+0000 7f28fd400640 1 -- 172.21.15.150:0/3195135801 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e802f820 con 0x7f2900072620 2024-04-15T15:26:06.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.553+0000 7f28fd400640 1 -- 172.21.15.150:0/3195135801 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e802f9c0 con 0x7f2900072620 2024-04-15T15:26:06.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.553+0000 7f28fd400640 1 -- 172.21.15.150:0/3195135801 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e802fd90 con 0x7f2900072620 2024-04-15T15:26:06.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.554+0000 7f28ff200640 1 -- 172.21.15.150:0/3195135801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2900072620 msgr2=0x7f290015cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.554+0000 7f28ff200640 1 --2- 172.21.15.150:0/3195135801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2900072620 0x7f290015cd10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f28e80098a0 tx=0x7f28e802ba00 comp rx=0 tx=0).stop 2024-04-15T15:26:06.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.554+0000 7f28ff200640 1 -- 172.21.15.150:0/3195135801 shutdown_connections 2024-04-15T15:26:06.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.554+0000 7f28ff200640 1 --2- 172.21.15.150:0/3195135801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f290015d250 0x7f290006f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.554+0000 7f28ff200640 1 --2- 172.21.15.150:0/3195135801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2900072620 0x7f290015cd10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.554+0000 7f28ff200640 1 -- 172.21.15.150:0/3195135801 >> 172.21.15.150:0/3195135801 conn(0x7f29000b6170 msgr2=0x7f290006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:06.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.554+0000 7f28ff200640 1 -- 172.21.15.150:0/3195135801 shutdown_connections 2024-04-15T15:26:06.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.554+0000 7f28ff200640 1 -- 172.21.15.150:0/3195135801 wait complete. 2024-04-15T15:26:06.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.556+0000 7f28ff200640 1 Processor -- start 2024-04-15T15:26:06.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.556+0000 7f28ff200640 1 -- start start 2024-04-15T15:26:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.557+0000 7f28ff200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2900072620 0x7f2900152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.557+0000 7f28ff200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29000cec60 0x7f2900141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.557+0000 7f28ff200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f290015d250 0x7f2900142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.557+0000 7f28ff200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f29001404d0 con 0x7f2900072620 2024-04-15T15:26:06.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.557+0000 7f28ff200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29001401f0 con 0x7f290015d250 2024-04-15T15:26:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.557+0000 7f28ff200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f290015d700 con 0x7f29000cec60 2024-04-15T15:26:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2900072620 0x7f2900152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2900072620 0x7f2900152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51132/0 (socket says 172.21.15.150:51132) 2024-04-15T15:26:06.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 -- 172.21.15.150:0/128488299 learned_addr learned my addr 172.21.15.150:0/128488299 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:06.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28f7e00640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29000cec60 0x7f2900141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fe800640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f290015d250 0x7f2900142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 -- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29000cec60 msgr2=0x7f2900141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29000cec60 0x7f2900141be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 -- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f290015d250 msgr2=0x7f2900142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f290015d250 0x7f2900142120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 -- 172.21.15.150:0/128488299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2900153e30 con 0x7f2900072620 2024-04-15T15:26:06.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fe800640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f290015d250 0x7f2900142120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.558+0000 7f28fde00640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2900072620 0x7f2900152760 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c400 tx=0x7f28f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.559+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f00052d0 con 0x7f2900072620 2024-04-15T15:26:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.559+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f0005490 con 0x7f2900072620 2024-04-15T15:26:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.559+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0007260 con 0x7f2900072620 2024-04-15T15:26:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.559+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2900154110 con 0x7f2900072620 2024-04-15T15:26:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.559+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29001546d0 con 0x7f2900072620 2024-04-15T15:26:06.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.560+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2900068b40 con 0x7f2900072620 2024-04-15T15:26:06.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.561+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f28f0007400 con 0x7f2900072620 2024-04-15T15:26:06.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.562+0000 7f28f6000640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28dc06c980 0x7f28dc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.562+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f28f008aeb0 con 0x7f2900072620 2024-04-15T15:26:06.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.562+0000 7f28f6000640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f28dc0728e0 0x7f28dc074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.562+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 --> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28f0009e50 con 0x7f28dc0728e0 2024-04-15T15:26:06.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.562+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f28f008d4d0 con 0x7f2900072620 2024-04-15T15:26:06.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.562+0000 7f28f7e00640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28dc06c980 0x7f28dc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.562+0000 7f28fe800640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f28dc0728e0 0x7f28dc074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.563+0000 7f28fe800640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f28dc0728e0 0x7f28dc074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.564+0000 7f28f7e00640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28dc06c980 0x7f28dc06ee40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28e8036000 tx=0x7f28e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.564+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 <== osd.2 v2:172.21.15.150:6808/1792706494 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28f0009e50 con 0x7f28dc0728e0 2024-04-15T15:26:06.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.578+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 --> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f29000730e0 con 0x7f28dc0728e0 2024-04-15T15:26:06.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.579+0000 7f28f6000640 1 -- 172.21.15.150:0/128488299 <== osd.2 v2:172.21.15.150:6808/1792706494 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f29000730e0 con 0x7f28dc0728e0 2024-04-15T15:26:06.581 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:26:06.581 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:26:06.581 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:26:06.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.579+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f28dc0728e0 msgr2=0x7f28dc074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.579+0000 7f28ff200640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.150:6808/1792706494,v1:172.21.15.150:6809/1792706494] conn(0x7f28dc0728e0 0x7f28dc074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28dc06c980 msgr2=0x7f28dc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28ff200640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f28dc06c980 0x7f28dc06ee40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28e8036000 tx=0x7f28e8008030 comp rx=0 tx=0).stop 2024-04-15T15:26:06.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2900072620 msgr2=0x7f2900152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28ff200640 1 --2- 172.21.15.150:0/128488299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2900072620 0x7f2900152760 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c400 tx=0x7f28f000c8c0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28fe800640 1 -- 172.21.15.150:0/128488299 reap_dead start 2024-04-15T15:26:06.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 shutdown_connections 2024-04-15T15:26:06.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 >> 172.21.15.150:0/128488299 conn(0x7f29000b6170 msgr2=0x7f290006de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:06.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 shutdown_connections 2024-04-15T15:26:06.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.580+0000 7f28ff200640 1 -- 172.21.15.150:0/128488299 wait complete. 2024-04-15T15:26:06.598 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-15T15:26:06.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.786+0000 7faa11a00640 1 Processor -- start 2024-04-15T15:26:06.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.786+0000 7faa11a00640 1 -- start start 2024-04-15T15:26:06.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa11a00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa0c0725b0 0x7faa0c06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa11a00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0c072a20 con 0x7faa0c072100 2024-04-15T15:26:06.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa11a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0c15d200 con 0x7faa0c13f6a0 2024-04-15T15:26:06.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa11a00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7faa0c0a95b0 con 0x7faa0c0725b0 2024-04-15T15:26:06.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa0b400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa0c0725b0 0x7faa0c06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa0b400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa0c0725b0 0x7faa0c06fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51136/0 (socket says 172.21.15.150:51136) 2024-04-15T15:26:06.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa0b400640 1 -- 172.21.15.150:0/2018936197 learned_addr learned my addr 172.21.15.150:0/2018936197 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:06.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa03e00640 1 --1- 172.21.15.150:0/2018936197 >> v1:172.21.15.150:6789/0 conn(0x7faa0c072100 0x7faa0c06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:38404/0 (socket says 172.21.15.150:38404) 2024-04-15T15:26:06.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa0b400640 1 -- 172.21.15.150:0/2018936197 >> v1:172.21.15.187:6789/0 conn(0x7faa0c13f6a0 legacy=0x7faa0c0ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:06.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa0b400640 1 -- 172.21.15.150:0/2018936197 >> v1:172.21.15.150:6789/0 conn(0x7faa0c072100 legacy=0x7faa0c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.788+0000 7faa0b400640 1 -- 172.21.15.150:0/2018936197 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c15d4e0 con 0x7faa0c0725b0 2024-04-15T15:26:06.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.790+0000 7faa0b400640 1 --2- 172.21.15.150:0/2018936197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c06fb00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faa04009e50 tx=0x7faa0402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1393e01855604b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.790+0000 7faa0a000640 1 -- 172.21.15.150:0/2018936197 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0402cbb0 con 0x7faa0c0725b0 2024-04-15T15:26:06.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.790+0000 7faa0a000640 1 -- 172.21.15.150:0/2018936197 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0402cd50 con 0x7faa0c0725b0 2024-04-15T15:26:06.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.790+0000 7faa0a000640 1 -- 172.21.15.150:0/2018936197 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04034610 con 0x7faa0c0725b0 2024-04-15T15:26:06.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.790+0000 7faa11a00640 1 -- 172.21.15.150:0/2018936197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 msgr2=0x7faa0c06fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.790+0000 7faa11a00640 1 --2- 172.21.15.150:0/2018936197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c06fb00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faa04009e50 tx=0x7faa0402c010 comp rx=0 tx=0).stop 2024-04-15T15:26:06.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.791+0000 7faa11a00640 1 -- 172.21.15.150:0/2018936197 shutdown_connections 2024-04-15T15:26:06.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.791+0000 7faa11a00640 1 --2- 172.21.15.150:0/2018936197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c06fb00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.791+0000 7faa11a00640 1 -- 172.21.15.150:0/2018936197 >> 172.21.15.150:0/2018936197 conn(0x7faa0c0b2460 msgr2=0x7faa0c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:06.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.791+0000 7faa11a00640 1 -- 172.21.15.150:0/2018936197 shutdown_connections 2024-04-15T15:26:06.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.791+0000 7faa11a00640 1 -- 172.21.15.150:0/2018936197 wait complete. 2024-04-15T15:26:06.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.792+0000 7faa11a00640 1 Processor -- start 2024-04-15T15:26:06.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.793+0000 7faa11a00640 1 -- start start 2024-04-15T15:26:06.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.793+0000 7faa11a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa0c072100 0x7faa0c156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa11a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa11a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa0c13f6a0 0x7faa0c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa11a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c15dd80 con 0x7faa0c0725b0 2024-04-15T15:26:06.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa11a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c13fef0 con 0x7faa0c13f6a0 2024-04-15T15:26:06.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa11a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c15ce60 con 0x7faa0c072100 2024-04-15T15:26:06.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa03e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa03e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c1586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51140/0 (socket says 172.21.15.150:51140) 2024-04-15T15:26:06.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa03e00640 1 -- 172.21.15.150:0/2567558265 learned_addr learned my addr 172.21.15.150:0/2567558265 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:06.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa0aa00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa0c072100 0x7faa0c156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa0b400640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa0c13f6a0 0x7faa0c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa03e00640 1 -- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa0c072100 msgr2=0x7faa0c156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa03e00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa0c072100 0x7faa0c156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa03e00640 1 -- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa0c13f6a0 msgr2=0x7faa0c159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:06.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa03e00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa0c13f6a0 0x7faa0c159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:06.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.794+0000 7faa03e00640 1 -- 172.21.15.150:0/2567558265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c137370 con 0x7faa0c0725b0 2024-04-15T15:26:06.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.795+0000 7faa03e00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c1586e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c3c0 tx=0x7fa9f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.795+0000 7faa02a00640 1 -- 172.21.15.150:0/2567558265 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40074e0 con 0x7faa0c0725b0 2024-04-15T15:26:06.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.795+0000 7faa02a00640 1 -- 172.21.15.150:0/2567558265 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f4007680 con 0x7faa0c0725b0 2024-04-15T15:26:06.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.795+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c15a4b0 con 0x7faa0c0725b0 2024-04-15T15:26:06.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.795+0000 7faa02a00640 1 -- 172.21.15.150:0/2567558265 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4004a00 con 0x7faa0c0725b0 2024-04-15T15:26:06.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.795+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c142250 con 0x7faa0c0725b0 2024-04-15T15:26:06.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.796+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c068c80 con 0x7faa0c0725b0 2024-04-15T15:26:06.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.797+0000 7faa02a00640 1 -- 172.21.15.150:0/2567558265 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa9f401a020 con 0x7faa0c0725b0 2024-04-15T15:26:06.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.798+0000 7faa02a00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9e806c9b0 0x7fa9e806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:06.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.798+0000 7faa02a00640 1 -- 172.21.15.150:0/2567558265 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fa9f408b150 con 0x7faa0c0725b0 2024-04-15T15:26:06.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.799+0000 7faa0aa00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9e806c9b0 0x7fa9e806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:06.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.800+0000 7faa0aa00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9e806c9b0 0x7fa9e806ee70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0015d0 tx=0x7fa9fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:06.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:06.803+0000 7faa02a00640 1 -- 172.21.15.150:0/2567558265 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9f4059730 con 0x7faa0c0725b0 2024-04-15T15:26:07.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.005+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7faa0c0adf10 con 0x7fa9e806c9b0 2024-04-15T15:26:07.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.007+0000 7faa02a00640 1 -- 172.21.15.150:0/2567558265 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faa0c0adf10 con 0x7fa9e806c9b0 2024-04-15T15:26:07.009 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 2 to deep-scrub 2024-04-15T15:26:07.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.009+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9e806c9b0 msgr2=0x7fa9e806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.009+0000 7faa11a00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9e806c9b0 0x7fa9e806ee70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0015d0 tx=0x7fa9fc009040 comp rx=0 tx=0).stop 2024-04-15T15:26:07.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.009+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 msgr2=0x7faa0c1586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.009+0000 7faa11a00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c1586e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c3c0 tx=0x7fa9f400c880 comp rx=0 tx=0).stop 2024-04-15T15:26:07.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.010+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 shutdown_connections 2024-04-15T15:26:07.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.010+0000 7faa11a00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa0c13f6a0 0x7faa0c159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.010+0000 7faa11a00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0725b0 0x7faa0c1586e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.010+0000 7faa11a00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa9e806c9b0 0x7fa9e806ee70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.010+0000 7faa11a00640 1 --2- 172.21.15.150:0/2567558265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa0c072100 0x7faa0c156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.010+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 >> 172.21.15.150:0/2567558265 conn(0x7faa0c0b2460 msgr2=0x7faa0c0ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:07.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.010+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 shutdown_connections 2024-04-15T15:26:07.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.010+0000 7faa11a00640 1 -- 172.21.15.150:0/2567558265 wait complete. 2024-04-15T15:26:07.031 INFO:tasks.ceph:Scrubbing osd.3 2024-04-15T15:26:07.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:26:07.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.246+0000 7fc116e00640 1 Processor -- start 2024-04-15T15:26:07.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.246+0000 7fc116e00640 1 -- start start 2024-04-15T15:26:07.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.248+0000 7fc116e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc11015d420 0x7fc11006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.248+0000 7fc116e00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11013fc70 con 0x7fc110072620 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.248+0000 7fc116e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc110141a00 con 0x7fc1100cec60 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.248+0000 7fc116e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fc11013f9e0 con 0x7fc11015d420 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.248+0000 7fc116400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc11015d420 0x7fc11006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.248+0000 7fc116400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc11015d420 0x7fc11006f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46320/0 (socket says 172.21.15.150:46320) 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.248+0000 7fc116400640 1 -- 172.21.15.150:0/741764404 learned_addr learned my addr 172.21.15.150:0/741764404 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.248+0000 7fc115000640 1 --1- 172.21.15.150:0/741764404 >> v1:172.21.15.150:6789/0 conn(0x7fc110072620 0x7fc11015cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42008/0 (socket says 172.21.15.150:42008) 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.249+0000 7fc116400640 1 -- 172.21.15.150:0/741764404 >> v1:172.21.15.187:6789/0 conn(0x7fc1100cec60 legacy=0x7fc110071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.249+0000 7fc116400640 1 -- 172.21.15.150:0/741764404 >> v1:172.21.15.150:6789/0 conn(0x7fc110072620 legacy=0x7fc11015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.249+0000 7fc116400640 1 -- 172.21.15.150:0/741764404 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11015da40 con 0x7fc11015d420 2024-04-15T15:26:07.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.250+0000 7fc116400640 1 --2- 172.21.15.150:0/741764404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11015d420 0x7fc11006f110 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc108009e50 tx=0x7fc10802bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae45b2420a8707a server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.250+0000 7fc10fe00640 1 -- 172.21.15.150:0/741764404 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10802cb60 con 0x7fc11015d420 2024-04-15T15:26:07.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.250+0000 7fc10fe00640 1 -- 172.21.15.150:0/741764404 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10802cd00 con 0x7fc11015d420 2024-04-15T15:26:07.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.250+0000 7fc10fe00640 1 -- 172.21.15.150:0/741764404 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108034610 con 0x7fc11015d420 2024-04-15T15:26:07.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.251+0000 7fc116e00640 1 -- 172.21.15.150:0/741764404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11015d420 msgr2=0x7fc11006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.251+0000 7fc116e00640 1 --2- 172.21.15.150:0/741764404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11015d420 0x7fc11006f110 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc108009e50 tx=0x7fc10802bfc0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.251+0000 7fc116e00640 1 -- 172.21.15.150:0/741764404 shutdown_connections 2024-04-15T15:26:07.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.251+0000 7fc116e00640 1 --2- 172.21.15.150:0/741764404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11015d420 0x7fc11006f110 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.251+0000 7fc116e00640 1 -- 172.21.15.150:0/741764404 >> 172.21.15.150:0/741764404 conn(0x7fc1100b6170 msgr2=0x7fc11006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:07.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.251+0000 7fc116e00640 1 -- 172.21.15.150:0/741764404 shutdown_connections 2024-04-15T15:26:07.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.251+0000 7fc116e00640 1 -- 172.21.15.150:0/741764404 wait complete. 2024-04-15T15:26:07.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.253+0000 7fc116e00640 1 Processor -- start 2024-04-15T15:26:07.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.253+0000 7fc116e00640 1 -- start start 2024-04-15T15:26:07.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc116e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc110072620 0x7fc11015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc116e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100cec60 0x7fc11015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc116e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11015d420 0x7fc1101574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc116e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101404d0 con 0x7fc1100cec60 2024-04-15T15:26:07.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc116e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101401f0 con 0x7fc110072620 2024-04-15T15:26:07.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc116e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc11006de40 con 0x7fc11015d420 2024-04-15T15:26:07.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc115000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100cec60 0x7fc11015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc115000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100cec60 0x7fc11015f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46322/0 (socket says 172.21.15.150:46322) 2024-04-15T15:26:07.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.254+0000 7fc115000640 1 -- 172.21.15.150:0/307405763 learned_addr learned my addr 172.21.15.150:0/307405763 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:07.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc116400640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11015d420 0x7fc1101574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc115a00640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc110072620 0x7fc11015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc115000640 1 -- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11015d420 msgr2=0x7fc1101574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc115000640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11015d420 0x7fc1101574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc115000640 1 -- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc110072620 msgr2=0x7fc11015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc115000640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc110072620 0x7fc11015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc115000640 1 -- 172.21.15.150:0/307405763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1100ae570 con 0x7fc1100cec60 2024-04-15T15:26:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc115a00640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc110072620 0x7fc11015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc116400640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11015d420 0x7fc1101574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc115000640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100cec60 0x7fc11015f410 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c3f0 tx=0x7fc0fc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0074e0 con 0x7fc1100cec60 2024-04-15T15:26:07.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc007680 con 0x7fc1100cec60 2024-04-15T15:26:07.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.255+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc110159550 con 0x7fc1100cec60 2024-04-15T15:26:07.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.256+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc004a00 con 0x7fc1100cec60 2024-04-15T15:26:07.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.256+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc110159b10 con 0x7fc1100cec60 2024-04-15T15:26:07.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.257+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1100ae850 con 0x7fc1100cec60 2024-04-15T15:26:07.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.258+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc0fc01a020 con 0x7fc1100cec60 2024-04-15T15:26:07.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.259+0000 7fc10d600640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc0f006c980 0x7fc0f006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.259+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fc0fc08b170 con 0x7fc1100cec60 2024-04-15T15:26:07.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.259+0000 7fc10d600640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7fc0f00728e0 0x7fc0f0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.259+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 --> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0f00753b0 con 0x7fc0f00728e0 2024-04-15T15:26:07.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.259+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc092050 con 0x7fc1100cec60 2024-04-15T15:26:07.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.259+0000 7fc116400640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7fc0f00728e0 0x7fc0f0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.259+0000 7fc115a00640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc0f006c980 0x7fc0f006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.260+0000 7fc116400640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7fc0f00728e0 0x7fc0f0074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.261+0000 7fc115a00640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc0f006c980 0x7fc0f006ee40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc104003eb0 tx=0x7fc104009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.261+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 <== osd.3 v2:172.21.15.150:6816/3354529328 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc0f00753b0 con 0x7fc0f00728e0 2024-04-15T15:26:07.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.276+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 --> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc110068b40 con 0x7fc0f00728e0 2024-04-15T15:26:07.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.276+0000 7fc10d600640 1 -- 172.21.15.150:0/307405763 <== osd.3 v2:172.21.15.150:6816/3354529328 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc110068b40 con 0x7fc0f00728e0 2024-04-15T15:26:07.279 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:26:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:26:07.280 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:26:07.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7fc0f00728e0 msgr2=0x7fc0f0074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.150:6816/3354529328,v1:172.21.15.150:6817/3354529328] conn(0x7fc0f00728e0 0x7fc0f0074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc0f006c980 msgr2=0x7fc0f006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fc0f006c980 0x7fc0f006ee40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc104003eb0 tx=0x7fc104009040 comp rx=0 tx=0).stop 2024-04-15T15:26:07.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100cec60 msgr2=0x7fc11015f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 --2- 172.21.15.150:0/307405763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100cec60 0x7fc11015f410 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c3f0 tx=0x7fc0fc00c8b0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116400640 1 -- 172.21.15.150:0/307405763 reap_dead start 2024-04-15T15:26:07.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 shutdown_connections 2024-04-15T15:26:07.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 >> 172.21.15.150:0/307405763 conn(0x7fc1100b6170 msgr2=0x7fc11015dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:07.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 shutdown_connections 2024-04-15T15:26:07.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.277+0000 7fc116e00640 1 -- 172.21.15.150:0/307405763 wait complete. 2024-04-15T15:26:07.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-15T15:26:07.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.490+0000 7efe3d400640 1 Processor -- start 2024-04-15T15:26:07.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.491+0000 7efe3d400640 1 -- start start 2024-04-15T15:26:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.492+0000 7efe3d400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe380afff0 0x7efe380cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.492+0000 7efe3d400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efe38144e40 0x7efe3815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.492+0000 7efe3d400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe380cfd50 con 0x7efe3815c130 2024-04-15T15:26:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.492+0000 7efe3d400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7efe380b3f30 con 0x7efe380afff0 2024-04-15T15:26:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.492+0000 7efe3d400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efe380b4210 con 0x7efe38144e40 2024-04-15T15:26:07.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.492+0000 7efe36800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe380afff0 0x7efe380cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.492+0000 7efe37200640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7efe3815c130 0x7efe3806f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42022/0 (socket says 172.21.15.150:42022) 2024-04-15T15:26:07.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.492+0000 7efe37200640 1 -- 172.21.15.150:0/100283672 learned_addr learned my addr 172.21.15.150:0/100283672 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:07.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.493+0000 7efe36800640 1 -- 172.21.15.150:0/100283672 >> v2:172.21.15.187:3300/0 conn(0x7efe38144e40 msgr2=0x7efe3815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:07.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.493+0000 7efe36800640 1 --2- 172.21.15.150:0/100283672 >> v2:172.21.15.187:3300/0 conn(0x7efe38144e40 0x7efe3815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.493+0000 7efe36800640 1 -- 172.21.15.150:0/100283672 >> v1:172.21.15.150:6789/0 conn(0x7efe3815c130 legacy=0x7efe3806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.493+0000 7efe36800640 1 -- 172.21.15.150:0/100283672 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe380d0030 con 0x7efe380afff0 2024-04-15T15:26:07.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.494+0000 7efe36800640 1 --2- 172.21.15.150:0/100283672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe380afff0 0x7efe380cf690 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efe380735a0 tx=0x7efe2802da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40108053f6711af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.494+0000 7efe35e00640 1 -- 172.21.15.150:0/100283672 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28031870 con 0x7efe380afff0 2024-04-15T15:26:07.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.494+0000 7efe35e00640 1 -- 172.21.15.150:0/100283672 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe28031a10 con 0x7efe380afff0 2024-04-15T15:26:07.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.494+0000 7efe35e00640 1 -- 172.21.15.150:0/100283672 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28031de0 con 0x7efe380afff0 2024-04-15T15:26:07.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.495+0000 7efe3d400640 1 -- 172.21.15.150:0/100283672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe380afff0 msgr2=0x7efe380cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.495+0000 7efe3d400640 1 --2- 172.21.15.150:0/100283672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe380afff0 0x7efe380cf690 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efe380735a0 tx=0x7efe2802da50 comp rx=0 tx=0).stop 2024-04-15T15:26:07.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.495+0000 7efe3d400640 1 -- 172.21.15.150:0/100283672 shutdown_connections 2024-04-15T15:26:07.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.495+0000 7efe3d400640 1 --2- 172.21.15.150:0/100283672 >> v2:172.21.15.187:3300/0 conn(0x7efe38144e40 0x7efe3815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.495+0000 7efe3d400640 1 --2- 172.21.15.150:0/100283672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe380afff0 0x7efe380cf690 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.495+0000 7efe3d400640 1 -- 172.21.15.150:0/100283672 >> 172.21.15.150:0/100283672 conn(0x7efe380b2460 msgr2=0x7efe380b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:07.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.495+0000 7efe3d400640 1 -- 172.21.15.150:0/100283672 shutdown_connections 2024-04-15T15:26:07.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.495+0000 7efe3d400640 1 -- 172.21.15.150:0/100283672 wait complete. 2024-04-15T15:26:07.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.497+0000 7efe3d400640 1 Processor -- start 2024-04-15T15:26:07.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.497+0000 7efe3d400640 1 -- start start 2024-04-15T15:26:07.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe3d400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe380afff0 0x7efe3815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe3d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe38144e40 0x7efe38141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe3d400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe3815c130 0x7efe380ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe3d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe38145420 con 0x7efe38144e40 2024-04-15T15:26:07.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe3d400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efe381452a0 con 0x7efe3815c130 2024-04-15T15:26:07.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe3d400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe3806f920 con 0x7efe380afff0 2024-04-15T15:26:07.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe2de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe38144e40 0x7efe38141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe2de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe38144e40 0x7efe38141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46352/0 (socket says 172.21.15.150:46352) 2024-04-15T15:26:07.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.498+0000 7efe2de00640 1 -- 172.21.15.150:0/1677722434 learned_addr learned my addr 172.21.15.150:0/1677722434 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:07.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe37200640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe3815c130 0x7efe380ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe36800640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe380afff0 0x7efe3815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2de00640 1 -- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe380afff0 msgr2=0x7efe3815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2de00640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe380afff0 0x7efe3815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2de00640 1 -- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe3815c130 msgr2=0x7efe380ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2de00640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe3815c130 0x7efe380ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2de00640 1 -- 172.21.15.150:0/1677722434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe381402b0 con 0x7efe38144e40 2024-04-15T15:26:07.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe36800640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe380afff0 0x7efe3815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:07.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2de00640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe38144e40 0x7efe38141350 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efe2000b4a0 tx=0x7efe2000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2f400640 1 -- 172.21.15.150:0/1677722434 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20014070 con 0x7efe38144e40 2024-04-15T15:26:07.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2f400640 1 -- 172.21.15.150:0/1677722434 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe20010610 con 0x7efe38144e40 2024-04-15T15:26:07.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe38140590 con 0x7efe38144e40 2024-04-15T15:26:07.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.499+0000 7efe2f400640 1 -- 172.21.15.150:0/1677722434 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20010a00 con 0x7efe38144e40 2024-04-15T15:26:07.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.500+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe38140b50 con 0x7efe38144e40 2024-04-15T15:26:07.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.501+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe38068c80 con 0x7efe38144e40 2024-04-15T15:26:07.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.502+0000 7efe2f400640 1 -- 172.21.15.150:0/1677722434 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7efe20017070 con 0x7efe38144e40 2024-04-15T15:26:07.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.502+0000 7efe2f400640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efe1406c980 0x7efe1406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.503+0000 7efe2f400640 1 -- 172.21.15.150:0/1677722434 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7efe2008b2d0 con 0x7efe38144e40 2024-04-15T15:26:07.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.503+0000 7efe36800640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efe1406c980 0x7efe1406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.504+0000 7efe36800640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efe1406c980 0x7efe1406ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efe28038000 tx=0x7efe28008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.507+0000 7efe2f400640 1 -- 172.21.15.150:0/1677722434 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe20059a40 con 0x7efe38144e40 2024-04-15T15:26:07.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.708+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7efe38140060 con 0x7efe1406c980 2024-04-15T15:26:07.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.709+0000 7efe2f400640 1 -- 172.21.15.150:0/1677722434 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efe38140060 con 0x7efe1406c980 2024-04-15T15:26:07.711 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 3 to deep-scrub 2024-04-15T15:26:07.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efe1406c980 msgr2=0x7efe1406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efe1406c980 0x7efe1406ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efe28038000 tx=0x7efe28008030 comp rx=0 tx=0).stop 2024-04-15T15:26:07.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe38144e40 msgr2=0x7efe38141350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe38144e40 0x7efe38141350 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efe2000b4a0 tx=0x7efe2000b960 comp rx=0 tx=0).stop 2024-04-15T15:26:07.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 shutdown_connections 2024-04-15T15:26:07.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe3815c130 0x7efe380ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe38144e40 0x7efe38141350 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7efe1406c980 0x7efe1406ee40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 --2- 172.21.15.150:0/1677722434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe380afff0 0x7efe3815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 >> 172.21.15.150:0/1677722434 conn(0x7efe380b2460 msgr2=0x7efe3806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:07.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 shutdown_connections 2024-04-15T15:26:07.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.712+0000 7efe3d400640 1 -- 172.21.15.150:0/1677722434 wait complete. 2024-04-15T15:26:07.732 INFO:tasks.ceph:Scrubbing osd.4 2024-04-15T15:26:07.732 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:26:07.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.949+0000 7f33ee400640 1 Processor -- start 2024-04-15T15:26:07.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.950+0000 7f33ee400640 1 -- start start 2024-04-15T15:26:07.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.951+0000 7f33ee400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e81462f0 0x7f33e8072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.951+0000 7f33ee400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f33e8072690 0x7f33e80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.951+0000 7f33ee400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33e80ce000 con 0x7f33e80cd680 2024-04-15T15:26:07.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.951+0000 7f33ee400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f33e80a9640 con 0x7f33e8072690 2024-04-15T15:26:07.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.951+0000 7f33ee400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f33e8072af0 con 0x7f33e81462f0 2024-04-15T15:26:07.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.952+0000 7f33e7e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f33e80cd680 0x7f33e806e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42032/0 (socket says 172.21.15.150:42032) 2024-04-15T15:26:07.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.952+0000 7f33e7e00640 1 -- 172.21.15.150:0/2948312190 learned_addr learned my addr 172.21.15.150:0/2948312190 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:07.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.952+0000 7f33e7400640 1 --2- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e81462f0 0x7f33e8072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.952+0000 7f33e6a00640 1 --2- 172.21.15.150:0/2948312190 >> v2:172.21.15.187:3300/0 conn(0x7f33e8072690 0x7f33e80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.952+0000 7f33e6000640 1 -- 172.21.15.150:0/2948312190 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 159688641 0 0) 0x7f33e80ce000 con 0x7f33e80cd680 2024-04-15T15:26:07.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.952+0000 7f33e6000640 1 -- 172.21.15.150:0/2948312190 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33c8003520 con 0x7f33e80cd680 2024-04-15T15:26:07.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.953+0000 7f33e6a00640 1 -- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e81462f0 msgr2=0x7f33e8072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:07.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.953+0000 7f33e6a00640 1 --2- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e81462f0 0x7f33e8072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.953+0000 7f33e6a00640 1 -- 172.21.15.150:0/2948312190 >> v1:172.21.15.150:6789/0 conn(0x7f33e80cd680 legacy=0x7f33e806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.953+0000 7f33e6000640 1 -- 172.21.15.150:0/2948312190 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3388830992 0 0) 0x7f33c8003520 con 0x7f33e80cd680 2024-04-15T15:26:07.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.953+0000 7f33e6a00640 1 -- 172.21.15.150:0/2948312190 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e80cdd20 con 0x7f33e8072690 2024-04-15T15:26:07.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.953+0000 7f33e7400640 1 --2- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e81462f0 0x7f33e8072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:26:07.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.954+0000 7f33e6a00640 1 --2- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8072690 0x7f33e80cd140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f33d00097a0 tx=0x7f33d002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a345858fc58c9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.954+0000 7f33e6000640 1 -- 172.21.15.150:0/2948312190 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0038070 con 0x7f33e8072690 2024-04-15T15:26:07.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.954+0000 7f33e6000640 1 -- 172.21.15.150:0/2948312190 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d0031500 con 0x7f33e8072690 2024-04-15T15:26:07.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.954+0000 7f33e6000640 1 -- 172.21.15.150:0/2948312190 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e00019f0 con 0x7f33e8072690 2024-04-15T15:26:07.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.955+0000 7f33ee400640 1 -- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8072690 msgr2=0x7f33e80cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.955+0000 7f33ee400640 1 --2- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8072690 0x7f33e80cd140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f33d00097a0 tx=0x7f33d002b820 comp rx=0 tx=0).stop 2024-04-15T15:26:07.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.955+0000 7f33ee400640 1 -- 172.21.15.150:0/2948312190 shutdown_connections 2024-04-15T15:26:07.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.955+0000 7f33ee400640 1 --2- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8072690 0x7f33e80cd140 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.955+0000 7f33ee400640 1 --2- 172.21.15.150:0/2948312190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e81462f0 0x7f33e8072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.955+0000 7f33ee400640 1 -- 172.21.15.150:0/2948312190 >> 172.21.15.150:0/2948312190 conn(0x7f33e80b2460 msgr2=0x7f33e80b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:07.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.955+0000 7f33ee400640 1 -- 172.21.15.150:0/2948312190 shutdown_connections 2024-04-15T15:26:07.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.955+0000 7f33ee400640 1 -- 172.21.15.150:0/2948312190 wait complete. 2024-04-15T15:26:07.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.957+0000 7f33ee400640 1 Processor -- start 2024-04-15T15:26:07.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.957+0000 7f33ee400640 1 -- start start 2024-04-15T15:26:07.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33ee400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e8072690 0x7f33e813f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33ee400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e80cd680 0x7f33e8152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33ee400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e81462f0 0x7f33e813ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33ee400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33e815d4d0 con 0x7f33e80cd680 2024-04-15T15:26:07.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33ee400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f33e815d1f0 con 0x7f33e81462f0 2024-04-15T15:26:07.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33ee400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f33e815ea00 con 0x7f33e8072690 2024-04-15T15:26:07.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33e6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e80cd680 0x7f33e8152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33e6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e80cd680 0x7f33e8152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46354/0 (socket says 172.21.15.150:46354) 2024-04-15T15:26:07.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.958+0000 7f33e6a00640 1 -- 172.21.15.150:0/3651668816 learned_addr learned my addr 172.21.15.150:0/3651668816 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:07.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e7e00640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e81462f0 0x7f33e813ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e7400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e8072690 0x7f33e813f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e6a00640 1 -- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e8072690 msgr2=0x7f33e813f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e6a00640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e8072690 0x7f33e813f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e6a00640 1 -- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e81462f0 msgr2=0x7f33e813ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e6a00640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e81462f0 0x7f33e813ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e6a00640 1 -- 172.21.15.150:0/3651668816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e813e7d0 con 0x7f33e80cd680 2024-04-15T15:26:07.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e7400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33e8072690 0x7f33e813f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:07.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.959+0000 7f33e6a00640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e80cd680 0x7f33e8152e80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f33d00098c0 tx=0x7f33d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.960+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0038040 con 0x7f33e80cd680 2024-04-15T15:26:07.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.960+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d0043070 con 0x7f33e80cd680 2024-04-15T15:26:07.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.960+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e813eab0 con 0x7f33e80cd680 2024-04-15T15:26:07.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.960+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0006e10 con 0x7f33e80cd680 2024-04-15T15:26:07.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.960+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e813f070 con 0x7f33e80cd680 2024-04-15T15:26:07.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.961+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33e8069260 con 0x7f33e80cd680 2024-04-15T15:26:07.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.962+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f33d003b070 con 0x7f33e80cd680 2024-04-15T15:26:07.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.963+0000 7f33df400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f33c406c980 0x7f33c406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.963+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f33d00acef0 con 0x7f33e80cd680 2024-04-15T15:26:07.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.963+0000 7f33df400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f33c40728e0 0x7f33c4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:07.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.963+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 --> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33d0035e90 con 0x7f33c40728e0 2024-04-15T15:26:07.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.963+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f33d00af510 con 0x7f33e80cd680 2024-04-15T15:26:07.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.963+0000 7f33e7400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f33c406c980 0x7f33c406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.964+0000 7f33e7e00640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f33c40728e0 0x7f33c4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:07.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.965+0000 7f33e7400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f33c406c980 0x7f33c406ee40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009a20 tx=0x7f33d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.965+0000 7f33e7e00640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f33c40728e0 0x7f33c4074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:07.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.967+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 <== osd.4 v2:172.21.15.187:6809/4034856676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33d0035e90 con 0x7f33c40728e0 2024-04-15T15:26:07.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.981+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 --> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f33e8068db0 con 0x7f33c40728e0 2024-04-15T15:26:07.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.982+0000 7f33df400640 1 -- 172.21.15.150:0/3651668816 <== osd.4 v2:172.21.15.187:6809/4034856676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f33e8068db0 con 0x7f33c40728e0 2024-04-15T15:26:07.984 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:26:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:26:07.984 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:26:07.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f33c40728e0 msgr2=0x7f33c4074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6809/4034856676,v1:172.21.15.187:6812/4034856676] conn(0x7f33c40728e0 0x7f33c4074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f33c406c980 msgr2=0x7f33c406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f33c406c980 0x7f33c406ee40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009a20 tx=0x7f33d8008040 comp rx=0 tx=0).stop 2024-04-15T15:26:07.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e80cd680 msgr2=0x7f33e8152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:07.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 --2- 172.21.15.150:0/3651668816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e80cd680 0x7f33e8152e80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f33d00098c0 tx=0x7f33d00046c0 comp rx=0 tx=0).stop 2024-04-15T15:26:07.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33e7e00640 1 -- 172.21.15.150:0/3651668816 reap_dead start 2024-04-15T15:26:07.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 shutdown_connections 2024-04-15T15:26:07.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 >> 172.21.15.150:0/3651668816 conn(0x7f33e80b2460 msgr2=0x7f33e81448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:07.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.983+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 shutdown_connections 2024-04-15T15:26:07.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:07.984+0000 7f33ee400640 1 -- 172.21.15.150:0/3651668816 wait complete. 2024-04-15T15:26:08.002 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-15T15:26:08.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.214+0000 7f23e2800640 1 Processor -- start 2024-04-15T15:26:08.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.215+0000 7f23e2800640 1 -- start start 2024-04-15T15:26:08.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.216+0000 7f23e2800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc1620b0 0x7f23dc0d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.216+0000 7f23e2800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc0d2ea0 con 0x7f23dc0cf7f0 2024-04-15T15:26:08.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.216+0000 7f23e2800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc0d3180 con 0x7f23dc06d590 2024-04-15T15:26:08.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.216+0000 7f23e2800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc0d3460 con 0x7f23dc1620b0 2024-04-15T15:26:08.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.216+0000 7f23e0a00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f23dc0cf7f0 0x7f23dc06ce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42036/0 (socket says 172.21.15.150:42036) 2024-04-15T15:26:08.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.216+0000 7f23e0a00640 1 -- 172.21.15.150:0/1404510762 learned_addr learned my addr 172.21.15.150:0/1404510762 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:08.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23e1400640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc1620b0 0x7f23dc0d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 985846452 0 0) 0x7f23dc0d2ea0 con 0x7f23dc0cf7f0 2024-04-15T15:26:08.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c4003520 con 0x7f23dc0cf7f0 2024-04-15T15:26:08.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 184518245 0 0) 0x7f23c4003520 con 0x7f23dc0cf7f0 2024-04-15T15:26:08.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc1620b0 msgr2=0x7f23dc0d3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:08.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc1620b0 0x7f23dc0d3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 >> v1:172.21.15.187:6789/0 conn(0x7f23dc06d590 legacy=0x7f23dc0b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc0d3740 con 0x7f23dc0cf7f0 2024-04-15T15:26:08.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f23c8001670 con 0x7f23dc0cf7f0 2024-04-15T15:26:08.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 >> v1:172.21.15.150:6789/0 conn(0x7f23dc0cf7f0 legacy=0x7f23dc06ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c40039e0 0x7f23c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23c4004800 0x7f23c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.217+0000 7f23dbe00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc1620b0 0x7f23c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc0d3460 con 0x7f23c4004800 2024-04-15T15:26:08.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23c400bb00 con 0x7f23c40039e0 2024-04-15T15:26:08.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23c400bd00 con 0x7f23dc1620b0 2024-04-15T15:26:08.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e0a00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23c4004800 0x7f23c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e1400640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c40039e0 0x7f23c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e1e00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc1620b0 0x7f23c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e0a00640 1 -- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc1620b0 msgr2=0x7f23c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:08.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e0a00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc1620b0 0x7f23c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e0a00640 1 -- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c40039e0 msgr2=0x7f23c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:08.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e0a00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c40039e0 0x7f23c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e0a00640 1 -- 172.21.15.150:0/1404510762 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c4003520 con 0x7f23c4004800 2024-04-15T15:26:08.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.218+0000 7f23e1400640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c40039e0 0x7f23c4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-15T15:26:08.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.219+0000 7f23e0a00640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23c4004800 0x7f23c4004c80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001eb0 tx=0x7f23c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abc325fa82f23d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8007d30 con 0x7f23c4004800 2024-04-15T15:26:08.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8030070 con 0x7f23c4004800 2024-04-15T15:26:08.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23dbe00640 1 -- 172.21.15.150:0/1404510762 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8005d30 con 0x7f23c4004800 2024-04-15T15:26:08.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23e2800640 1 -- 172.21.15.150:0/1404510762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23c4004800 msgr2=0x7f23c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23e2800640 1 --2- 172.21.15.150:0/1404510762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23c4004800 0x7f23c4004c80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001eb0 tx=0x7f23c4005830 comp rx=0 tx=0).stop 2024-04-15T15:26:08.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23e1e00640 1 -- 172.21.15.150:0/1404510762 reap_dead start 2024-04-15T15:26:08.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23e2800640 1 -- 172.21.15.150:0/1404510762 shutdown_connections 2024-04-15T15:26:08.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23e2800640 1 -- 172.21.15.150:0/1404510762 >> 172.21.15.150:0/1404510762 conn(0x7f23dc0d18d0 msgr2=0x7f23dc0d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:08.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.220+0000 7f23e2800640 1 -- 172.21.15.150:0/1404510762 shutdown_connections 2024-04-15T15:26:08.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.221+0000 7f23e2800640 1 -- 172.21.15.150:0/1404510762 wait complete. 2024-04-15T15:26:08.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.222+0000 7f23e2800640 1 Processor -- start 2024-04-15T15:26:08.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.222+0000 7f23e2800640 1 -- start start 2024-04-15T15:26:08.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.223+0000 7f23e2800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c4004800 0x7f23dc1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.223+0000 7f23e2800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc15f6e0 0x7f23dc0c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.223+0000 7f23e2800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc150ae0 0x7f23dc0d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.223+0000 7f23e2800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc0d5550 con 0x7f23dc15f6e0 2024-04-15T15:26:08.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.223+0000 7f23e2800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc162490 con 0x7f23c4004800 2024-04-15T15:26:08.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.223+0000 7f23e2800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc0d56d0 con 0x7f23dc150ae0 2024-04-15T15:26:08.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e0a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc15f6e0 0x7f23dc0c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e0a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc15f6e0 0x7f23dc0c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46370/0 (socket says 172.21.15.150:46370) 2024-04-15T15:26:08.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e0a00640 1 -- 172.21.15.150:0/2869448306 learned_addr learned my addr 172.21.15.150:0/2869448306 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:08.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e1400640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c4004800 0x7f23dc1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e1e00640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc150ae0 0x7f23dc0d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e0a00640 1 -- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc150ae0 msgr2=0x7f23dc0d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e0a00640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc150ae0 0x7f23dc0d3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e0a00640 1 -- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c4004800 msgr2=0x7f23dc1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e0a00640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c4004800 0x7f23dc1f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e0a00640 1 -- 172.21.15.150:0/2869448306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc1612d0 con 0x7f23dc15f6e0 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e1400640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c4004800 0x7f23dc1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.224+0000 7f23e1e00640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc150ae0 0x7f23dc0d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.225+0000 7f23e0a00640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc15f6e0 0x7f23dc0c7a40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f23c800c300 tx=0x7f23c8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.225+0000 7f23d9600640 1 -- 172.21.15.150:0/2869448306 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8045070 con 0x7f23dc15f6e0 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.225+0000 7f23d9600640 1 -- 172.21.15.150:0/2869448306 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8000ed0 con 0x7f23dc15f6e0 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.225+0000 7f23d9600640 1 -- 172.21.15.150:0/2869448306 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8004df0 con 0x7f23dc15f6e0 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.225+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc1615b0 con 0x7f23dc15f6e0 2024-04-15T15:26:08.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.225+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc0c7f80 con 0x7f23dc15f6e0 2024-04-15T15:26:08.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.226+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc0c9d60 con 0x7f23dc15f6e0 2024-04-15T15:26:08.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.227+0000 7f23d9600640 1 -- 172.21.15.150:0/2869448306 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f23c8007070 con 0x7f23dc15f6e0 2024-04-15T15:26:08.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.228+0000 7f23d9600640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f23bc06c980 0x7f23bc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.229+0000 7f23d9600640 1 -- 172.21.15.150:0/2869448306 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f23c80aca30 con 0x7f23dc15f6e0 2024-04-15T15:26:08.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.229+0000 7f23e1400640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f23bc06c980 0x7f23bc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.230+0000 7f23e1400640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f23bc06c980 0x7f23bc06ee40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009a80 tx=0x7f23d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.233+0000 7f23d9600640 1 -- 172.21.15.150:0/2869448306 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23c807aa40 con 0x7f23dc15f6e0 2024-04-15T15:26:08.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.432+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f23dc06daf0 con 0x7f23bc06c980 2024-04-15T15:26:08.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.433+0000 7f23d9600640 1 -- 172.21.15.150:0/2869448306 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f23dc06daf0 con 0x7f23bc06c980 2024-04-15T15:26:08.435 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 4 to deep-scrub 2024-04-15T15:26:08.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f23bc06c980 msgr2=0x7f23bc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f23bc06c980 0x7f23bc06ee40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009a80 tx=0x7f23d0008040 comp rx=0 tx=0).stop 2024-04-15T15:26:08.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc15f6e0 msgr2=0x7f23dc0c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc15f6e0 0x7f23dc0c7a40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f23c800c300 tx=0x7f23c8001a40 comp rx=0 tx=0).stop 2024-04-15T15:26:08.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 shutdown_connections 2024-04-15T15:26:08.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f23bc06c980 0x7f23bc06ee40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23dc150ae0 0x7f23dc0d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc15f6e0 0x7f23dc0c7a40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 --2- 172.21.15.150:0/2869448306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c4004800 0x7f23dc1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 >> 172.21.15.150:0/2869448306 conn(0x7f23dc0d18d0 msgr2=0x7f23dc0b8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:08.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 shutdown_connections 2024-04-15T15:26:08.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.436+0000 7f23e2800640 1 -- 172.21.15.150:0/2869448306 wait complete. 2024-04-15T15:26:08.457 INFO:tasks.ceph:Scrubbing osd.5 2024-04-15T15:26:08.457 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:26:08.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.669+0000 7f678f200640 1 Processor -- start 2024-04-15T15:26:08.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.669+0000 7f678f200640 1 -- start start 2024-04-15T15:26:08.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.671+0000 7f678f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67900cec60 0x7f6790071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.671+0000 7f678f200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f679015d420 0x7f679006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.671+0000 7f678f200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679013fc70 con 0x7f6790072450 2024-04-15T15:26:08.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.671+0000 7f678f200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f679013f9e0 con 0x7f679015d420 2024-04-15T15:26:08.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.671+0000 7f678f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6790072890 con 0x7f67900cec60 2024-04-15T15:26:08.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.671+0000 7f678e800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f679015d420 0x7f679006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.671+0000 7f678e800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f679015d420 0x7f679006f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46382/0 (socket says 172.21.15.150:46382) 2024-04-15T15:26:08.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.672+0000 7f678e800640 1 -- 172.21.15.150:0/3679285288 learned_addr learned my addr 172.21.15.150:0/3679285288 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:08.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.672+0000 7f678de00640 1 --2- 172.21.15.150:0/3679285288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67900cec60 0x7f6790071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.672+0000 7f678e800640 1 -- 172.21.15.150:0/3679285288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67900cec60 msgr2=0x7f6790071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:08.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.672+0000 7f678e800640 1 --2- 172.21.15.150:0/3679285288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67900cec60 0x7f6790071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.672+0000 7f678e800640 1 -- 172.21.15.150:0/3679285288 >> v1:172.21.15.150:6789/0 conn(0x7f6790072450 legacy=0x7f679015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.672+0000 7f678e800640 1 -- 172.21.15.150:0/3679285288 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6790141a00 con 0x7f679015d420 2024-04-15T15:26:08.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.673+0000 7f678e800640 1 --2- 172.21.15.150:0/3679285288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f679015d420 0x7f679006f110 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6788009e50 tx=0x7f678802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbde7e05a8d04da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.673+0000 7f678d400640 1 -- 172.21.15.150:0/3679285288 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678802cb90 con 0x7f679015d420 2024-04-15T15:26:08.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.673+0000 7f678d400640 1 -- 172.21.15.150:0/3679285288 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678802cd30 con 0x7f679015d420 2024-04-15T15:26:08.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.673+0000 7f678d400640 1 -- 172.21.15.150:0/3679285288 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788034610 con 0x7f679015d420 2024-04-15T15:26:08.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.674+0000 7f678f200640 1 -- 172.21.15.150:0/3679285288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f679015d420 msgr2=0x7f679006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.674+0000 7f678f200640 1 --2- 172.21.15.150:0/3679285288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f679015d420 0x7f679006f110 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6788009e50 tx=0x7f678802bff0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.674+0000 7f678f200640 1 -- 172.21.15.150:0/3679285288 shutdown_connections 2024-04-15T15:26:08.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.674+0000 7f678f200640 1 --2- 172.21.15.150:0/3679285288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f679015d420 0x7f679006f110 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.674+0000 7f678f200640 1 --2- 172.21.15.150:0/3679285288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67900cec60 0x7f6790071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.674+0000 7f678f200640 1 -- 172.21.15.150:0/3679285288 >> 172.21.15.150:0/3679285288 conn(0x7f67900b6170 msgr2=0x7f679006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:08.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.674+0000 7f678f200640 1 -- 172.21.15.150:0/3679285288 shutdown_connections 2024-04-15T15:26:08.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.674+0000 7f678f200640 1 -- 172.21.15.150:0/3679285288 wait complete. 2024-04-15T15:26:08.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.676+0000 7f678f200640 1 Processor -- start 2024-04-15T15:26:08.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.676+0000 7f678f200640 1 -- start start 2024-04-15T15:26:08.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f678f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790072450 0x7f6790157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f678f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67900cec60 0x7f679015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f678f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679015d420 0x7f67901f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f678f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67901401f0 con 0x7f67900cec60 2024-04-15T15:26:08.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f678f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f679013ff10 con 0x7f679015d420 2024-04-15T15:26:08.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f678f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67901404d0 con 0x7f6790072450 2024-04-15T15:26:08.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f6785400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67900cec60 0x7f679015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f6785400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67900cec60 0x7f679015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46388/0 (socket says 172.21.15.150:46388) 2024-04-15T15:26:08.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.677+0000 7f6785400640 1 -- 172.21.15.150:0/1175723346 learned_addr learned my addr 172.21.15.150:0/1175723346 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:08.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f678de00640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790072450 0x7f6790157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f6785400640 1 -- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790072450 msgr2=0x7f6790157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f6785400640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790072450 0x7f6790157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f6785400640 1 -- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679015d420 msgr2=0x7f67901f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f678e800640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679015d420 0x7f67901f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f6785400640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679015d420 0x7f67901f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f6785400640 1 -- 172.21.15.150:0/1175723346 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6790157910 con 0x7f67900cec60 2024-04-15T15:26:08.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f678de00640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790072450 0x7f6790157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:08.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.678+0000 7f6785400640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67900cec60 0x7f679015a0d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f677800c430 tx=0x7f677800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.679+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67780052d0 con 0x7f67900cec60 2024-04-15T15:26:08.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.679+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6778005490 con 0x7f67900cec60 2024-04-15T15:26:08.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.679+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6790157bf0 con 0x7f67900cec60 2024-04-15T15:26:08.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.679+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778007260 con 0x7f67900cec60 2024-04-15T15:26:08.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.679+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67901581b0 con 0x7f67900cec60 2024-04-15T15:26:08.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.680+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f679015a610 con 0x7f67900cec60 2024-04-15T15:26:08.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.681+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6778007400 con 0x7f67900cec60 2024-04-15T15:26:08.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.682+0000 7f6786a00640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f676c06c980 0x7f676c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.682+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f677808aec0 con 0x7f67900cec60 2024-04-15T15:26:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.682+0000 7f6786a00640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f676c0728e0 0x7f676c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.682+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 --> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6778009e50 con 0x7f676c0728e0 2024-04-15T15:26:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.682+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f677808d4e0 con 0x7f67900cec60 2024-04-15T15:26:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.682+0000 7f678de00640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f676c06c980 0x7f676c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.683+0000 7f678e800640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f676c0728e0 0x7f676c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.683+0000 7f678de00640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f676c06c980 0x7f676c06ee40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.684+0000 7f678e800640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f676c0728e0 0x7f676c074d00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.686+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 <== osd.5 v2:172.21.15.187:6800/2717020412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6778009e50 con 0x7f676c0728e0 2024-04-15T15:26:08.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.701+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 --> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f67901585f0 con 0x7f676c0728e0 2024-04-15T15:26:08.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f6786a00640 1 -- 172.21.15.150:0/1175723346 <== osd.5 v2:172.21.15.187:6800/2717020412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f67901585f0 con 0x7f676c0728e0 2024-04-15T15:26:08.703 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:26:08.704 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:26:08.704 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:26:08.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f676c0728e0 msgr2=0x7f676c074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678f200640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6800/2717020412,v1:172.21.15.187:6801/2717020412] conn(0x7f676c0728e0 0x7f676c074d00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f676c06c980 msgr2=0x7f676c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678f200640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f676c06c980 0x7f676c06ee40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780008040 comp rx=0 tx=0).stop 2024-04-15T15:26:08.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67900cec60 msgr2=0x7f679015a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678f200640 1 --2- 172.21.15.150:0/1175723346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67900cec60 0x7f679015a0d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f677800c430 tx=0x7f677800c8f0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678e800640 1 -- 172.21.15.150:0/1175723346 reap_dead start 2024-04-15T15:26:08.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 shutdown_connections 2024-04-15T15:26:08.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.702+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 >> 172.21.15.150:0/1175723346 conn(0x7f67900b6170 msgr2=0x7f67900af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:08.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.703+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 shutdown_connections 2024-04-15T15:26:08.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.703+0000 7f678f200640 1 -- 172.21.15.150:0/1175723346 wait complete. 2024-04-15T15:26:08.723 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-15T15:26:08.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.933+0000 7fea4b200640 1 Processor -- start 2024-04-15T15:26:08.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.933+0000 7fea4b200640 1 -- start start 2024-04-15T15:26:08.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea4b200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fea4c0b0cb0 0x7fea4c0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea4b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c15d770 0x7fea4c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea4b200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4c0cd940 con 0x7fea4c0b59b0 2024-04-15T15:26:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea4b200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fea4c0ce550 con 0x7fea4c0b0cb0 2024-04-15T15:26:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea4b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea4c0cdc20 con 0x7fea4c15d770 2024-04-15T15:26:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea4a800640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fea4c0b59b0 0x7fea4c15dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42054/0 (socket says 172.21.15.150:42054) 2024-04-15T15:26:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea4a800640 1 -- 172.21.15.150:0/3696352357 learned_addr learned my addr 172.21.15.150:0/3696352357 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea43e00640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c15d770 0x7fea4c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.935+0000 7fea49e00640 1 --2- 172.21.15.150:0/3696352357 >> v2:172.21.15.187:3300/0 conn(0x7fea4c0b0cb0 0x7fea4c0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122477536 0 0) 0x7fea4c0cd940 con 0x7fea4c0b59b0 2024-04-15T15:26:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea30003520 con 0x7fea4c0b59b0 2024-04-15T15:26:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 105255303 0 0) 0x7fea30003520 con 0x7fea4c0b59b0 2024-04-15T15:26:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c15d770 msgr2=0x7fea4c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c15d770 0x7fea4c0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 >> v2:172.21.15.187:3300/0 conn(0x7fea4c0b0cb0 msgr2=0x7fea4c0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 --2- 172.21.15.150:0/3696352357 >> v2:172.21.15.187:3300/0 conn(0x7fea4c0b0cb0 0x7fea4c0ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c0cdf00 con 0x7fea4c0b59b0 2024-04-15T15:26:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fea440019f0 con 0x7fea4c0b59b0 2024-04-15T15:26:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 >> v1:172.21.15.150:6789/0 conn(0x7fea4c0b59b0 legacy=0x7fea4c15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea30004800 0x7fea30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c15d770 0x7fea300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c0ce550 con 0x7fea300039b0 2024-04-15T15:26:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c0cdc20 con 0x7fea30004800 2024-04-15T15:26:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea3000b9c0 con 0x7fea4c15d770 2024-04-15T15:26:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.936+0000 7fea49e00640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.937+0000 7fea43e00640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea30004800 0x7fea30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.937+0000 7fea49e00640 1 -- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c15d770 msgr2=0x7fea300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:08.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.937+0000 7fea49e00640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c15d770 0x7fea300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.937+0000 7fea49e00640 1 -- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea30004800 msgr2=0x7fea30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:08.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.937+0000 7fea49e00640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea30004800 0x7fea30004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.937+0000 7fea49e00640 1 -- 172.21.15.150:0/3696352357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea30003520 con 0x7fea300039b0 2024-04-15T15:26:08.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.938+0000 7fea49e00640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea30003dd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009ac0 tx=0x7fea30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3efbde0f1acb31 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.938+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c039070 con 0x7fea300039b0 2024-04-15T15:26:08.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.938+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3c02c710 con 0x7fea300039b0 2024-04-15T15:26:08.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.938+0000 7fea49400640 1 -- 172.21.15.150:0/3696352357 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c009690 con 0x7fea300039b0 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.939+0000 7fea4b200640 1 -- 172.21.15.150:0/3696352357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 msgr2=0x7fea30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.939+0000 7fea4b200640 1 --2- 172.21.15.150:0/3696352357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea30003dd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009ac0 tx=0x7fea30005830 comp rx=0 tx=0).stop 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.939+0000 7fea4a800640 1 -- 172.21.15.150:0/3696352357 reap_dead start 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.939+0000 7fea4b200640 1 -- 172.21.15.150:0/3696352357 shutdown_connections 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.939+0000 7fea4b200640 1 -- 172.21.15.150:0/3696352357 >> 172.21.15.150:0/3696352357 conn(0x7fea4c0b23a0 msgr2=0x7fea4c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.939+0000 7fea4b200640 1 -- 172.21.15.150:0/3696352357 shutdown_connections 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.939+0000 7fea4b200640 1 -- 172.21.15.150:0/3696352357 wait complete. 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.941+0000 7fea4b200640 1 Processor -- start 2024-04-15T15:26:08.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.941+0000 7fea4b200640 1 -- start start 2024-04-15T15:26:08.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea4b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea4c146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea4b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c155dd0 0x7fea4c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea4b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea4c0b40b0 0x7fea4c06e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea4b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c0cef20 con 0x7fea300039b0 2024-04-15T15:26:08.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea4b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c0b1090 con 0x7fea4c0b40b0 2024-04-15T15:26:08.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea4b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea4c147030 con 0x7fea4c155dd0 2024-04-15T15:26:08.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea49e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea4c146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea49e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea4c146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46398/0 (socket says 172.21.15.150:46398) 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea49e00640 1 -- 172.21.15.150:0/1363510805 learned_addr learned my addr 172.21.15.150:0/1363510805 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea43e00640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c155dd0 0x7fea4c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea49e00640 1 -- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c155dd0 msgr2=0x7fea4c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea49e00640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c155dd0 0x7fea4c155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea49e00640 1 -- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea4c0b40b0 msgr2=0x7fea4c06e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea49e00640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea4c0b40b0 0x7fea4c06e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.942+0000 7fea49e00640 1 -- 172.21.15.150:0/1363510805 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c156300 con 0x7fea300039b0 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.943+0000 7fea49e00640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea4c146af0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fea3c000c00 tx=0x7fea3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.943+0000 7fea42000640 1 -- 172.21.15.150:0/1363510805 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c039050 con 0x7fea300039b0 2024-04-15T15:26:08.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.943+0000 7fea42000640 1 -- 172.21.15.150:0/1363510805 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3c045070 con 0x7fea300039b0 2024-04-15T15:26:08.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.943+0000 7fea42000640 1 -- 172.21.15.150:0/1363510805 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0044e0 con 0x7fea300039b0 2024-04-15T15:26:08.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.943+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c154630 con 0x7fea300039b0 2024-04-15T15:26:08.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.943+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c154bf0 con 0x7fea300039b0 2024-04-15T15:26:08.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.944+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4c0755c0 con 0x7fea300039b0 2024-04-15T15:26:08.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.945+0000 7fea42000640 1 -- 172.21.15.150:0/1363510805 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fea3c003a50 con 0x7fea300039b0 2024-04-15T15:26:08.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.946+0000 7fea42000640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fea2806c9b0 0x7fea2806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:08.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.946+0000 7fea42000640 1 -- 172.21.15.150:0/1363510805 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fea3c0ad730 con 0x7fea300039b0 2024-04-15T15:26:08.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.947+0000 7fea43e00640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fea2806c9b0 0x7fea2806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:08.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.948+0000 7fea43e00640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fea2806c9b0 0x7fea2806ee70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea4c14f1a0 tx=0x7fea34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:08.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:08.951+0000 7fea42000640 1 -- 172.21.15.150:0/1363510805 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea3c07bd10 con 0x7fea300039b0 2024-04-15T15:26:09.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.152+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fea4c151b30 con 0x7fea2806c9b0 2024-04-15T15:26:09.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.154+0000 7fea42000640 1 -- 172.21.15.150:0/1363510805 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fea4c151b30 con 0x7fea2806c9b0 2024-04-15T15:26:09.155 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 5 to deep-scrub 2024-04-15T15:26:09.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fea2806c9b0 msgr2=0x7fea2806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fea2806c9b0 0x7fea2806ee70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea4c14f1a0 tx=0x7fea34008040 comp rx=0 tx=0).stop 2024-04-15T15:26:09.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 msgr2=0x7fea4c146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea4c146af0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fea3c000c00 tx=0x7fea3c004730 comp rx=0 tx=0).stop 2024-04-15T15:26:09.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 shutdown_connections 2024-04-15T15:26:09.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea4c0b40b0 0x7fea4c06e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fea2806c9b0 0x7fea2806ee70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea4c155dd0 0x7fea4c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.156+0000 7fea4b200640 1 --2- 172.21.15.150:0/1363510805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea300039b0 0x7fea4c146af0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.157+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 >> 172.21.15.150:0/1363510805 conn(0x7fea4c0b23a0 msgr2=0x7fea4c15cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:09.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.157+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 shutdown_connections 2024-04-15T15:26:09.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.157+0000 7fea4b200640 1 -- 172.21.15.150:0/1363510805 wait complete. 2024-04-15T15:26:09.177 INFO:tasks.ceph:Scrubbing osd.6 2024-04-15T15:26:09.178 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:26:09.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.392+0000 7f2489a00640 1 Processor -- start 2024-04-15T15:26:09.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.392+0000 7f2489a00640 1 -- start start 2024-04-15T15:26:09.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.394+0000 7f2489a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f248406f660 0x7f24840cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.394+0000 7f2489a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f24840cd9b0 0x7f248415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.394+0000 7f2489a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24840b3cf0 con 0x7f2484144e40 2024-04-15T15:26:09.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.394+0000 7f2489a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f24840b3fd0 con 0x7f24840cd9b0 2024-04-15T15:26:09.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.394+0000 7f2489a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f24840b42b0 con 0x7f248406f660 2024-04-15T15:26:09.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.394+0000 7f247b400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f248406f660 0x7f24840cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.394+0000 7f2489000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f24840cd9b0 0x7f248415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.395+0000 7f2489000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f24840cd9b0 0x7f248415bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.150:33492/0 (socket says 172.21.15.150:33492) 2024-04-15T15:26:09.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.395+0000 7f2489000640 1 -- 172.21.15.150:0/658561527 learned_addr learned my addr 172.21.15.150:0/658561527 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:09.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.395+0000 7f247b400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f248406f660 0x7f24840cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.150:47554/0 (socket says 172.21.15.150:47554) 2024-04-15T15:26:09.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.395+0000 7f2489000640 1 -- 172.21.15.150:0/658561527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f248406f660 msgr2=0x7f24840cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:09.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.395+0000 7f2489000640 1 --2- 172.21.15.150:0/658561527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f248406f660 0x7f24840cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.396+0000 7f2489000640 1 -- 172.21.15.150:0/658561527 >> v1:172.21.15.187:6789/0 conn(0x7f2484144e40 legacy=0x7f248406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.396+0000 7f2489000640 1 -- 172.21.15.150:0/658561527 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24840b4590 con 0x7f24840cd9b0 2024-04-15T15:26:09.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.396+0000 7f247b400640 1 --2- 172.21.15.150:0/658561527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f248406f660 0x7f24840cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-15T15:26:09.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.397+0000 7f2489000640 1 --2- 172.21.15.150:0/658561527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24840cd9b0 0x7f248415bad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f247c009e50 tx=0x7f247c02bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e385593d0143be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:09.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2483400640 1 -- 172.21.15.150:0/658561527 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c02ca60 con 0x7f24840cd9b0 2024-04-15T15:26:09.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2483400640 1 -- 172.21.15.150:0/658561527 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c02cc00 con 0x7f24840cd9b0 2024-04-15T15:26:09.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2483400640 1 -- 172.21.15.150:0/658561527 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0343f0 con 0x7f24840cd9b0 2024-04-15T15:26:09.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2489a00640 1 -- 172.21.15.150:0/658561527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24840cd9b0 msgr2=0x7f248415bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2489a00640 1 --2- 172.21.15.150:0/658561527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24840cd9b0 0x7f248415bad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f247c009e50 tx=0x7f247c02bec0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2489a00640 1 -- 172.21.15.150:0/658561527 shutdown_connections 2024-04-15T15:26:09.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2489a00640 1 --2- 172.21.15.150:0/658561527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24840cd9b0 0x7f248415bad0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2489a00640 1 --2- 172.21.15.150:0/658561527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f248406f660 0x7f24840cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2489a00640 1 -- 172.21.15.150:0/658561527 >> 172.21.15.150:0/658561527 conn(0x7f24840b6170 msgr2=0x7f248406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:09.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.398+0000 7f2489a00640 1 -- 172.21.15.150:0/658561527 shutdown_connections 2024-04-15T15:26:09.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.399+0000 7f2489a00640 1 -- 172.21.15.150:0/658561527 wait complete. 2024-04-15T15:26:09.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.400+0000 7f2489a00640 1 Processor -- start 2024-04-15T15:26:09.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.400+0000 7f2489a00640 1 -- start start 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.401+0000 7f2489a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f248406f660 0x7f248413f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.401+0000 7f2489a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24840cd9b0 0x7f248406e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.401+0000 7f2489a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2484144e40 0x7f248406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.401+0000 7f2489a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2484145220 con 0x7f24840cd9b0 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.401+0000 7f2489a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24840cde20 con 0x7f248406f660 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.401+0000 7f2489a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f24840b4d90 con 0x7f2484144e40 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f247b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24840cd9b0 0x7f248406e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f247b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24840cd9b0 0x7f248406e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46406/0 (socket says 172.21.15.150:46406) 2024-04-15T15:26:09.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f247b400640 1 -- 172.21.15.150:0/3279791890 learned_addr learned my addr 172.21.15.150:0/3279791890 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:09.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f2489000640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2484144e40 0x7f248406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f2483e00640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f248406f660 0x7f248413f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f247b400640 1 -- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2484144e40 msgr2=0x7f248406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f247b400640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2484144e40 0x7f248406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f247b400640 1 -- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f248406f660 msgr2=0x7f248413f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f247b400640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f248406f660 0x7f248413f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.402+0000 7f247b400640 1 -- 172.21.15.150:0/3279791890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24841f2600 con 0x7f24840cd9b0 2024-04-15T15:26:09.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.403+0000 7f247b400640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24840cd9b0 0x7f248406e420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f246c00ebe0 tx=0x7f246c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:09.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.403+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c017070 con 0x7f24840cd9b0 2024-04-15T15:26:09.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.403+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c00cc30 con 0x7f24840cd9b0 2024-04-15T15:26:09.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.403+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24841f28e0 con 0x7f24840cd9b0 2024-04-15T15:26:09.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.403+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c013470 con 0x7f24840cd9b0 2024-04-15T15:26:09.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.403+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24841f2ea0 con 0x7f24840cd9b0 2024-04-15T15:26:09.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.404+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f248406c5a0 con 0x7f24840cd9b0 2024-04-15T15:26:09.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.405+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f246c01a070 con 0x7f24840cd9b0 2024-04-15T15:26:09.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.406+0000 7f2480c00640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f246006c980 0x7f246006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.406+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f246c08bd00 con 0x7f24840cd9b0 2024-04-15T15:26:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.406+0000 7f2480c00640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f24600728e0 0x7f2460074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.406+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 --> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24600753b0 con 0x7f24600728e0 2024-04-15T15:26:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.406+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f246c08e320 con 0x7f24840cd9b0 2024-04-15T15:26:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.407+0000 7f2483e00640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f246006c980 0x7f246006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.407+0000 7f2489000640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f24600728e0 0x7f2460074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.408+0000 7f2483e00640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f246006c980 0x7f246006ee40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2474007fd0 tx=0x7f2474009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:09.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.408+0000 7f2489000640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f24600728e0 0x7f2460074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.410+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 <== osd.6 v2:172.21.15.187:6811/455912281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24600753b0 con 0x7f24600728e0 2024-04-15T15:26:09.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.425+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 --> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2484068b40 con 0x7f24600728e0 2024-04-15T15:26:09.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2480c00640 1 -- 172.21.15.150:0/3279791890 <== osd.6 v2:172.21.15.187:6811/455912281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2484068b40 con 0x7f24600728e0 2024-04-15T15:26:09.428 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:26:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:26:09.428 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:26:09.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f24600728e0 msgr2=0x7f2460074d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489a00640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6811/455912281,v1:172.21.15.187:6814/455912281] conn(0x7f24600728e0 0x7f2460074d00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f246006c980 msgr2=0x7f246006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489a00640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f246006c980 0x7f246006ee40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2474007fd0 tx=0x7f2474009040 comp rx=0 tx=0).stop 2024-04-15T15:26:09.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24840cd9b0 msgr2=0x7f248406e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489a00640 1 --2- 172.21.15.150:0/3279791890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24840cd9b0 0x7f248406e420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f246c00ebe0 tx=0x7f246c00c520 comp rx=0 tx=0).stop 2024-04-15T15:26:09.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489000640 1 -- 172.21.15.150:0/3279791890 reap_dead start 2024-04-15T15:26:09.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 shutdown_connections 2024-04-15T15:26:09.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.426+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 >> 172.21.15.150:0/3279791890 conn(0x7f24840b6170 msgr2=0x7f248406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:09.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.427+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 shutdown_connections 2024-04-15T15:26:09.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.427+0000 7f2489a00640 1 -- 172.21.15.150:0/3279791890 wait complete. 2024-04-15T15:26:09.445 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-15T15:26:09.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.660+0000 7f82dd400640 1 Processor -- start 2024-04-15T15:26:09.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.660+0000 7f82dd400640 1 -- start start 2024-04-15T15:26:09.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.662+0000 7f82dd400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d80d19b0 0x7f82d80d39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.662+0000 7f82dd400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d80b78f0 con 0x7f82d815eab0 2024-04-15T15:26:09.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.662+0000 7f82dd400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d8162d40 con 0x7f82d80d1550 2024-04-15T15:26:09.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.662+0000 7f82dd400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82d80d2e50 con 0x7f82d80d19b0 2024-04-15T15:26:09.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.662+0000 7f82d7e00640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f82d815eab0 0x7f82d8162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:42060/0 (socket says 172.21.15.150:42060) 2024-04-15T15:26:09.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.662+0000 7f82d7e00640 1 -- 172.21.15.150:0/2570616696 learned_addr learned my addr 172.21.15.150:0/2570616696 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:09.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82dca00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d80d19b0 0x7f82d80d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034212979 0 0) 0x7f82c8001490 con 0x7f82d815eab0 2024-04-15T15:26:09.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82bc003520 con 0x7f82d815eab0 2024-04-15T15:26:09.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 689004530 0 0) 0x7f82bc003520 con 0x7f82d815eab0 2024-04-15T15:26:09.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d80d19b0 msgr2=0x7f82d80d39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:09.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d80d19b0 0x7f82d80d39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 >> v1:172.21.15.187:6789/0 conn(0x7f82d80d1550 legacy=0x7f82d80d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d8163010 con 0x7f82d815eab0 2024-04-15T15:26:09.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7f82c8001bc0 con 0x7f82d815eab0 2024-04-15T15:26:09.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 >> v1:172.21.15.150:6789/0 conn(0x7f82d815eab0 legacy=0x7f82d8162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d80d19b0 0x7f82bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82bc003f60 0x7f82bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82bc004d40 0x7f82bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f82d80d2e50 con 0x7f82bc004d40 2024-04-15T15:26:09.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc00baa0 con 0x7f82bc003f60 2024-04-15T15:26:09.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.663+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc00bca0 con 0x7f82d80d19b0 2024-04-15T15:26:09.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82dca00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82bc004d40 0x7f82bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82d7e00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d80d19b0 0x7f82bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82d7400640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82bc003f60 0x7f82bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82dca00640 1 -- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d80d19b0 msgr2=0x7f82bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:09.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82dca00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d80d19b0 0x7f82bc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82dca00640 1 -- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82bc003f60 msgr2=0x7f82bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:09.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82dca00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82bc003f60 0x7f82bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82dca00640 1 -- 172.21.15.150:0/2570616696 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc003520 con 0x7f82bc004d40 2024-04-15T15:26:09.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.664+0000 7f82d7400640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82bc003f60 0x7f82bc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:09.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.665+0000 7f82dca00640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82bc004d40 0x7f82bc0051c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009ac0 tx=0x7f82bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420f916a8fead2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:09.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0039070 con 0x7f82bc004d40 2024-04-15T15:26:09.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d002c6b0 con 0x7f82bc004d40 2024-04-15T15:26:09.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82d6a00640 1 -- 172.21.15.150:0/2570616696 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0009690 con 0x7f82bc004d40 2024-04-15T15:26:09.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82dd400640 1 -- 172.21.15.150:0/2570616696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82bc004d40 msgr2=0x7f82bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82dd400640 1 --2- 172.21.15.150:0/2570616696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82bc004d40 0x7f82bc0051c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009ac0 tx=0x7f82bc005830 comp rx=0 tx=0).stop 2024-04-15T15:26:09.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82dca00640 1 -- 172.21.15.150:0/2570616696 reap_dead start 2024-04-15T15:26:09.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82dd400640 1 -- 172.21.15.150:0/2570616696 shutdown_connections 2024-04-15T15:26:09.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82dd400640 1 -- 172.21.15.150:0/2570616696 >> 172.21.15.150:0/2570616696 conn(0x7f82d8073240 msgr2=0x7f82d8073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82dd400640 1 -- 172.21.15.150:0/2570616696 shutdown_connections 2024-04-15T15:26:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.666+0000 7f82dd400640 1 -- 172.21.15.150:0/2570616696 wait complete. 2024-04-15T15:26:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.668+0000 7f82dd400640 1 Processor -- start 2024-04-15T15:26:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.668+0000 7f82dd400640 1 -- start start 2024-04-15T15:26:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dd400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82bc004d40 0x7f82d8158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dd400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d80ce310 0x7f82d815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dd400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82d815cd20 0x7f82d81f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dd400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f82d80d1e40 con 0x7f82d815cd20 2024-04-15T15:26:09.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dd400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f82d80d2ae0 con 0x7f82d80ce310 2024-04-15T15:26:09.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dd400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82d815ee90 con 0x7f82bc004d40 2024-04-15T15:26:09.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82d815cd20 0x7f82d81f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dca00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82d815cd20 0x7f82d81f4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46424/0 (socket says 172.21.15.150:46424) 2024-04-15T15:26:09.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.669+0000 7f82dca00640 1 -- 172.21.15.150:0/1190126452 learned_addr learned my addr 172.21.15.150:0/1190126452 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:09.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82d7400640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d80ce310 0x7f82d815c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82d7e00640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82bc004d40 0x7f82d8158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82dca00640 1 -- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82bc004d40 msgr2=0x7f82d8158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82dca00640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82bc004d40 0x7f82d8158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82dca00640 1 -- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d80ce310 msgr2=0x7f82d815c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82dca00640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d80ce310 0x7f82d815c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82dca00640 1 -- 172.21.15.150:0/1190126452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d8159040 con 0x7f82d815cd20 2024-04-15T15:26:09.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82d7400640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d80ce310 0x7f82d815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:09.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82dca00640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82d815cd20 0x7f82d81f4a20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009be0 tx=0x7f82d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:09.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82cfe00640 1 -- 172.21.15.150:0/1190126452 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0044070 con 0x7f82d815cd20 2024-04-15T15:26:09.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.670+0000 7f82cfe00640 1 -- 172.21.15.150:0/1190126452 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d003a040 con 0x7f82d815cd20 2024-04-15T15:26:09.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.671+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d8158b60 con 0x7f82d815cd20 2024-04-15T15:26:09.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.671+0000 7f82cfe00640 1 -- 172.21.15.150:0/1190126452 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0039070 con 0x7f82d815cd20 2024-04-15T15:26:09.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.671+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d80cc580 con 0x7f82d815cd20 2024-04-15T15:26:09.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.672+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d80755c0 con 0x7f82d815cd20 2024-04-15T15:26:09.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.673+0000 7f82cfe00640 1 -- 172.21.15.150:0/1190126452 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f82d0047070 con 0x7f82d815cd20 2024-04-15T15:26:09.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.674+0000 7f82cfe00640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f82b806c9b0 0x7f82b806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:09.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.674+0000 7f82cfe00640 1 -- 172.21.15.150:0/1190126452 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f82d00ac2a0 con 0x7f82d815cd20 2024-04-15T15:26:09.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.674+0000 7f82d7e00640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f82b806c9b0 0x7f82b806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:09.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.675+0000 7f82d7e00640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f82b806c9b0 0x7f82b806ee70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f82c8001630 tx=0x7f82c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:09.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.678+0000 7f82cfe00640 1 -- 172.21.15.150:0/1190126452 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82d8159040 con 0x7f82d815cd20 2024-04-15T15:26:09.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.879+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f82d80d2460 con 0x7f82b806c9b0 2024-04-15T15:26:09.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.880+0000 7f82cfe00640 1 -- 172.21.15.150:0/1190126452 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f82d80d2460 con 0x7f82b806c9b0 2024-04-15T15:26:09.882 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 6 to deep-scrub 2024-04-15T15:26:09.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.882+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f82b806c9b0 msgr2=0x7f82b806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.882+0000 7f82dd400640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f82b806c9b0 0x7f82b806ee70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f82c8001630 tx=0x7f82c8009040 comp rx=0 tx=0).stop 2024-04-15T15:26:09.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.882+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82d815cd20 msgr2=0x7f82d81f4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:09.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.882+0000 7f82dd400640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82d815cd20 0x7f82d81f4a20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009be0 tx=0x7f82d0004730 comp rx=0 tx=0).stop 2024-04-15T15:26:09.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.883+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 shutdown_connections 2024-04-15T15:26:09.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.883+0000 7f82dd400640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82d815cd20 0x7f82d81f4a20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.883+0000 7f82dd400640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d80ce310 0x7f82d815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.883+0000 7f82dd400640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f82b806c9b0 0x7f82b806ee70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.883+0000 7f82dd400640 1 --2- 172.21.15.150:0/1190126452 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82bc004d40 0x7f82d8158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:09.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.883+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 >> 172.21.15.150:0/1190126452 conn(0x7f82d8073240 msgr2=0x7f82d815f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:09.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.883+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 shutdown_connections 2024-04-15T15:26:09.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:09.883+0000 7f82dd400640 1 -- 172.21.15.150:0/1190126452 wait complete. 2024-04-15T15:26:09.904 INFO:tasks.ceph:Scrubbing osd.7 2024-04-15T15:26:09.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-15T15:26:10.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.122+0000 7fa8d2e00640 1 Processor -- start 2024-04-15T15:26:10.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.122+0000 7fa8d2e00640 1 -- start start 2024-04-15T15:26:10.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.124+0000 7fa8d2e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b2140 0x7fa8cc0b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.124+0000 7fa8d2e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa8cc0b46a0 0x7fa8cc0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.124+0000 7fa8d2e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa8cc15fd50 0x7fa8cc0d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.124+0000 7fa8d2e00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fa8cc15cae0 con 0x7fa8cc15fd50 2024-04-15T15:26:10.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.124+0000 7fa8d2e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa8cc0b4e60 con 0x7fa8cc0b46a0 2024-04-15T15:26:10.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.124+0000 7fa8d2e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8cc0b4b80 con 0x7fa8cc0b2140 2024-04-15T15:26:10.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8d0e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa8cc15fd50 0x7fa8cc0d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8cbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b2140 0x7fa8cc0b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8d0e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa8cc15fd50 0x7fa8cc0d20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46428/0 (socket says 172.21.15.150:46428) 2024-04-15T15:26:10.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8d0e00640 1 -- 172.21.15.150:0/2698148521 learned_addr learned my addr 172.21.15.150:0/2698148521 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:10.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8c3400640 1 --2- 172.21.15.150:0/2698148521 >> v2:172.21.15.187:3300/0 conn(0x7fa8cc0b46a0 0x7fa8cc0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8d0e00640 1 -- 172.21.15.150:0/2698148521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b2140 msgr2=0x7fa8cc0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:10.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8d0e00640 1 --2- 172.21.15.150:0/2698148521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b2140 0x7fa8cc0b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8d0e00640 1 -- 172.21.15.150:0/2698148521 >> v2:172.21.15.187:3300/0 conn(0x7fa8cc0b46a0 msgr2=0x7fa8cc0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:10.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8d0e00640 1 --2- 172.21.15.150:0/2698148521 >> v2:172.21.15.187:3300/0 conn(0x7fa8cc0b46a0 0x7fa8cc0d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8d0e00640 1 -- 172.21.15.150:0/2698148521 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc15f270 con 0x7fa8cc15fd50 2024-04-15T15:26:10.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.125+0000 7fa8c3400640 1 --2- 172.21.15.150:0/2698148521 >> v2:172.21.15.187:3300/0 conn(0x7fa8cc0b46a0 0x7fa8cc0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:10.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.126+0000 7fa8d0e00640 1 --2- 172.21.15.150:0/2698148521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 0x7fa8cc0d20a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc06c9a0 tx=0x7fa8c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d577770dd7b88d server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.127+0000 7fa8cb400640 1 -- 172.21.15.150:0/2698148521 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4035020 con 0x7fa8cc15fd50 2024-04-15T15:26:10.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.127+0000 7fa8cb400640 1 -- 172.21.15.150:0/2698148521 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c4031870 con 0x7fa8cc15fd50 2024-04-15T15:26:10.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.127+0000 7fa8cb400640 1 -- 172.21.15.150:0/2698148521 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4031c40 con 0x7fa8cc15fd50 2024-04-15T15:26:10.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.127+0000 7fa8d2e00640 1 -- 172.21.15.150:0/2698148521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 msgr2=0x7fa8cc0d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.127+0000 7fa8d2e00640 1 --2- 172.21.15.150:0/2698148521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 0x7fa8cc0d20a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc06c9a0 tx=0x7fa8c402b910 comp rx=0 tx=0).stop 2024-04-15T15:26:10.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.128+0000 7fa8d2e00640 1 -- 172.21.15.150:0/2698148521 shutdown_connections 2024-04-15T15:26:10.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.128+0000 7fa8d2e00640 1 --2- 172.21.15.150:0/2698148521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 0x7fa8cc0d20a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.128+0000 7fa8d2e00640 1 --2- 172.21.15.150:0/2698148521 >> v2:172.21.15.187:3300/0 conn(0x7fa8cc0b46a0 0x7fa8cc0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.128+0000 7fa8d2e00640 1 --2- 172.21.15.150:0/2698148521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b2140 0x7fa8cc0b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.128+0000 7fa8d2e00640 1 -- 172.21.15.150:0/2698148521 >> 172.21.15.150:0/2698148521 conn(0x7fa8cc09aa40 msgr2=0x7fa8cc09ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:10.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.128+0000 7fa8d2e00640 1 -- 172.21.15.150:0/2698148521 shutdown_connections 2024-04-15T15:26:10.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.128+0000 7fa8d2e00640 1 -- 172.21.15.150:0/2698148521 wait complete. 2024-04-15T15:26:10.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.129+0000 7fa8d2e00640 1 Processor -- start 2024-04-15T15:26:10.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.130+0000 7fa8d2e00640 1 -- start start 2024-04-15T15:26:10.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.130+0000 7fa8d2e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b2140 0x7fa8cc150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d2e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0b46a0 0x7fa8cc153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d2e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 0x7fa8cc1616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d2e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc0b3050 con 0x7fa8cc15fd50 2024-04-15T15:26:10.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d2e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc15f830 con 0x7fa8cc0b46a0 2024-04-15T15:26:10.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d2e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8cc0b31d0 con 0x7fa8cc0b2140 2024-04-15T15:26:10.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 0x7fa8cc1616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 0x7fa8cc1616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46440/0 (socket says 172.21.15.150:46440) 2024-04-15T15:26:10.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d0e00640 1 -- 172.21.15.150:0/1015745828 learned_addr learned my addr 172.21.15.150:0/1015745828 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:10.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d0e00640 1 -- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b2140 msgr2=0x7fa8cc150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d0e00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b2140 0x7fa8cc150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d0e00640 1 -- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0b46a0 msgr2=0x7fa8cc153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d0e00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0b46a0 0x7fa8cc153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.131+0000 7fa8d0e00640 1 -- 172.21.15.150:0/1015745828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc150a40 con 0x7fa8cc15fd50 2024-04-15T15:26:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.132+0000 7fa8d0e00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 0x7fa8cc1616c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40090b0 tx=0x7fa8c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.132+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4040070 con 0x7fa8cc15fd50 2024-04-15T15:26:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.132+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c4030d00 con 0x7fa8cc15fd50 2024-04-15T15:26:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.132+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4035040 con 0x7fa8cc15fd50 2024-04-15T15:26:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.132+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc154620 con 0x7fa8cc15fd50 2024-04-15T15:26:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.132+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc153d80 con 0x7fa8cc15fd50 2024-04-15T15:26:10.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.133+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8cc068db0 con 0x7fa8cc15fd50 2024-04-15T15:26:10.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.134+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa8c4038020 con 0x7fa8cc15fd50 2024-04-15T15:26:10.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.135+0000 7fa8c8c00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8a806c9b0 0x7fa8a806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.135+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fa8c40ad610 con 0x7fa8cc15fd50 2024-04-15T15:26:10.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.135+0000 7fa8c8c00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7fa8a8072910 0x7fa8a8074d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.135+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 --> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8a80753e0 con 0x7fa8a8072910 2024-04-15T15:26:10.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.135+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa8c40b4050 con 0x7fa8cc15fd50 2024-04-15T15:26:10.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.136+0000 7fa8cbe00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8a806c9b0 0x7fa8a806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.136+0000 7fa8c3400640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7fa8a8072910 0x7fa8a8074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.137+0000 7fa8cbe00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8a806c9b0 0x7fa8a806ee70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00e110 tx=0x7fa8bc00d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.137+0000 7fa8c3400640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7fa8a8072910 0x7fa8a8074d30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.141 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.139+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 <== osd.7 v2:172.21.15.187:6808/2186687773 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8a80753e0 con 0x7fa8a8072910 2024-04-15T15:26:10.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.154+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 --> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa8cc0d1940 con 0x7fa8a8072910 2024-04-15T15:26:10.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.155+0000 7fa8c8c00640 1 -- 172.21.15.150:0/1015745828 <== osd.7 v2:172.21.15.187:6808/2186687773 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa8cc0d1940 con 0x7fa8a8072910 2024-04-15T15:26:10.157 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-15T15:26:10.157 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-15T15:26:10.157 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-15T15:26:10.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.155+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7fa8a8072910 msgr2=0x7fa8a8074d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.155+0000 7fa8d2e00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6808/2186687773,v1:172.21.15.187:6810/2186687773] conn(0x7fa8a8072910 0x7fa8a8074d30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.155+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8a806c9b0 msgr2=0x7fa8a806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.155+0000 7fa8d2e00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fa8a806c9b0 0x7fa8a806ee70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00e110 tx=0x7fa8bc00d040 comp rx=0 tx=0).stop 2024-04-15T15:26:10.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.155+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 msgr2=0x7fa8cc1616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.155+0000 7fa8d2e00640 1 --2- 172.21.15.150:0/1015745828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8cc15fd50 0x7fa8cc1616c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40090b0 tx=0x7fa8c40046c0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.156+0000 7fa8d0e00640 1 -- 172.21.15.150:0/1015745828 reap_dead start 2024-04-15T15:26:10.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.156+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 shutdown_connections 2024-04-15T15:26:10.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.156+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 >> 172.21.15.150:0/1015745828 conn(0x7fa8cc09aa40 msgr2=0x7fa8cc0ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:10.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.156+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 shutdown_connections 2024-04-15T15:26:10.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.156+0000 7fa8d2e00640 1 -- 172.21.15.150:0/1015745828 wait complete. 2024-04-15T15:26:10.175 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-15T15:26:10.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.387+0000 7fb484800640 1 Processor -- start 2024-04-15T15:26:10.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.388+0000 7fb484800640 1 -- start start 2024-04-15T15:26:10.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.389+0000 7fb484800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb47c162e90 0x7fb47c0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.389+0000 7fb484800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb47c0afd70 0x7fb47c072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.389+0000 7fb484800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 0x7fb47c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.389+0000 7fb484800640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7fb47c072a70 con 0x7fb47c162e90 2024-04-15T15:26:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.389+0000 7fb484800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb47c0b01d0 con 0x7fb47c0afd70 2024-04-15T15:26:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.389+0000 7fb484800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb47c0b3900 con 0x7fb47c072640 2024-04-15T15:26:10.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.389+0000 7fb481e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb47c162e90 0x7fb47c0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.390+0000 7fb481e00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb47c162e90 0x7fb47c0b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46442/0 (socket says 172.21.15.150:46442) 2024-04-15T15:26:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.390+0000 7fb481e00640 1 -- 172.21.15.150:0/2496669552 learned_addr learned my addr 172.21.15.150:0/2496669552 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.390+0000 7fb482800640 1 --2- 172.21.15.150:0/2496669552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 0x7fb47c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.390+0000 7fb481e00640 1 -- 172.21.15.150:0/2496669552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 msgr2=0x7fb47c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.390+0000 7fb481e00640 1 --2- 172.21.15.150:0/2496669552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 0x7fb47c06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.390+0000 7fb481e00640 1 -- 172.21.15.150:0/2496669552 >> v2:172.21.15.187:3300/0 conn(0x7fb47c0afd70 msgr2=0x7fb47c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:10.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.390+0000 7fb481e00640 1 --2- 172.21.15.150:0/2496669552 >> v2:172.21.15.187:3300/0 conn(0x7fb47c0afd70 0x7fb47c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.390+0000 7fb481e00640 1 -- 172.21.15.150:0/2496669552 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c0b3be0 con 0x7fb47c162e90 2024-04-15T15:26:10.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.391+0000 7fb481e00640 1 --2- 172.21.15.150:0/2496669552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c162e90 0x7fb47c0b58e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb47c074400 tx=0x7fb47002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b01cc30da16879 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.391+0000 7fb480a00640 1 -- 172.21.15.150:0/2496669552 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47002e070 con 0x7fb47c162e90 2024-04-15T15:26:10.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb480a00640 1 -- 172.21.15.150:0/2496669552 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb470037070 con 0x7fb47c162e90 2024-04-15T15:26:10.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb480a00640 1 -- 172.21.15.150:0/2496669552 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470031a00 con 0x7fb47c162e90 2024-04-15T15:26:10.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 -- 172.21.15.150:0/2496669552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c162e90 msgr2=0x7fb47c0b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 --2- 172.21.15.150:0/2496669552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c162e90 0x7fb47c0b58e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb47c074400 tx=0x7fb47002b880 comp rx=0 tx=0).stop 2024-04-15T15:26:10.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 -- 172.21.15.150:0/2496669552 shutdown_connections 2024-04-15T15:26:10.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 --2- 172.21.15.150:0/2496669552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 0x7fb47c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 --2- 172.21.15.150:0/2496669552 >> v2:172.21.15.187:3300/0 conn(0x7fb47c0afd70 0x7fb47c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 --2- 172.21.15.150:0/2496669552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c162e90 0x7fb47c0b58e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 -- 172.21.15.150:0/2496669552 >> 172.21.15.150:0/2496669552 conn(0x7fb47c0b2460 msgr2=0x7fb47c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:10.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 -- 172.21.15.150:0/2496669552 shutdown_connections 2024-04-15T15:26:10.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.392+0000 7fb484800640 1 -- 172.21.15.150:0/2496669552 wait complete. 2024-04-15T15:26:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.394+0000 7fb484800640 1 Processor -- start 2024-04-15T15:26:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.394+0000 7fb484800640 1 -- start start 2024-04-15T15:26:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.395+0000 7fb484800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 0x7fb47c0abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.395+0000 7fb484800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c0afd70 0x7fb47c0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.395+0000 7fb484800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb47c162e90 0x7fb47c0ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.395+0000 7fb484800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c06e640 con 0x7fb47c0afd70 2024-04-15T15:26:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.395+0000 7fb484800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c0b41a0 con 0x7fb47c162e90 2024-04-15T15:26:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.395+0000 7fb484800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb47c06e7c0 con 0x7fb47c072640 2024-04-15T15:26:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.395+0000 7fb481400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c0afd70 0x7fb47c0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c0afd70 0x7fb47c0adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46448/0 (socket says 172.21.15.150:46448) 2024-04-15T15:26:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481400640 1 -- 172.21.15.150:0/2340066894 learned_addr learned my addr 172.21.15.150:0/2340066894 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb482800640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb47c162e90 0x7fb47c0ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481e00640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 0x7fb47c0abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481400640 1 -- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 msgr2=0x7fb47c0abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481400640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 0x7fb47c0abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481400640 1 -- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb47c162e90 msgr2=0x7fb47c0ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481400640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb47c162e90 0x7fb47c0ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481400640 1 -- 172.21.15.150:0/2340066894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c0ae8a0 con 0x7fb47c0afd70 2024-04-15T15:26:10.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.396+0000 7fb481400640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c0afd70 0x7fb47c0adae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb46800b4a0 tx=0x7fb46800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.397+0000 7fb476000640 1 -- 172.21.15.150:0/2340066894 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468014070 con 0x7fb47c0afd70 2024-04-15T15:26:10.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.397+0000 7fb476000640 1 -- 172.21.15.150:0/2340066894 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb468010610 con 0x7fb47c0afd70 2024-04-15T15:26:10.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.397+0000 7fb476000640 1 -- 172.21.15.150:0/2340066894 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468010a00 con 0x7fb47c0afd70 2024-04-15T15:26:10.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.397+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c0aeb80 con 0x7fb47c0afd70 2024-04-15T15:26:10.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.397+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c0ae560 con 0x7fb47c0afd70 2024-04-15T15:26:10.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.398+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c0aef70 con 0x7fb47c0afd70 2024-04-15T15:26:10.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.399+0000 7fb476000640 1 -- 172.21.15.150:0/2340066894 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb468017070 con 0x7fb47c0afd70 2024-04-15T15:26:10.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.400+0000 7fb476000640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb45c06c980 0x7fb45c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.401+0000 7fb476000640 1 -- 172.21.15.150:0/2340066894 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fb46808b480 con 0x7fb47c0afd70 2024-04-15T15:26:10.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.401+0000 7fb481e00640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb45c06c980 0x7fb45c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.402+0000 7fb481e00640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb45c06c980 0x7fb45c06ee40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb470036000 tx=0x7fb47003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.405+0000 7fb476000640 1 -- 172.21.15.150:0/2340066894 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb468059a60 con 0x7fb47c0afd70 2024-04-15T15:26:10.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.601+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb47c0af5c0 con 0x7fb45c06c980 2024-04-15T15:26:10.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.602+0000 7fb476000640 1 -- 172.21.15.150:0/2340066894 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb47c0af5c0 con 0x7fb45c06c980 2024-04-15T15:26:10.604 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 7 to deep-scrub 2024-04-15T15:26:10.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.604+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb45c06c980 msgr2=0x7fb45c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.604+0000 7fb484800640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb45c06c980 0x7fb45c06ee40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb470036000 tx=0x7fb47003d040 comp rx=0 tx=0).stop 2024-04-15T15:26:10.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.604+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c0afd70 msgr2=0x7fb47c0adae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.604+0000 7fb484800640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c0afd70 0x7fb47c0adae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb46800b4a0 tx=0x7fb46800b960 comp rx=0 tx=0).stop 2024-04-15T15:26:10.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.604+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 shutdown_connections 2024-04-15T15:26:10.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.604+0000 7fb484800640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb47c162e90 0x7fb47c0ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.604+0000 7fb484800640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb47c0afd70 0x7fb47c0adae0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.604+0000 7fb484800640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fb45c06c980 0x7fb45c06ee40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.605+0000 7fb484800640 1 --2- 172.21.15.150:0/2340066894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb47c072640 0x7fb47c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.605+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 >> 172.21.15.150:0/2340066894 conn(0x7fb47c0b2460 msgr2=0x7fb47c15d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:10.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.605+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 shutdown_connections 2024-04-15T15:26:10.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.605+0000 7fb484800640 1 -- 172.21.15.150:0/2340066894 wait complete. 2024-04-15T15:26:10.625 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:26:10.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.828+0000 7f91af200640 1 Processor -- start 2024-04-15T15:26:10.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.828+0000 7f91af200640 1 -- start start 2024-04-15T15:26:10.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.829+0000 7f91af200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f91b00cee00 0x7f91b006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.829+0000 7f91af200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f91b015d2f0 0x7f91b00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.829+0000 7f91af200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b015dbb0 con 0x7f91b0071f50 2024-04-15T15:26:10.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.830+0000 7f91af200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f91b006f4f0 con 0x7f91b015d2f0 2024-04-15T15:26:10.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.830+0000 7f91af200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f91b00728f0 con 0x7f91b00cee00 2024-04-15T15:26:10.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.830+0000 7f91ae800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f91b015d2f0 0x7f91b00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.830+0000 7f91ae800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f91b015d2f0 0x7f91b00723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46460/0 (socket says 172.21.15.150:46460) 2024-04-15T15:26:10.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.830+0000 7f91ae800640 1 -- 172.21.15.150:0/4226720328 learned_addr learned my addr 172.21.15.150:0/4226720328 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:10.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.830+0000 7f91ade00640 1 --2- 172.21.15.150:0/4226720328 >> v2:172.21.15.187:3300/0 conn(0x7f91b00cee00 0x7f91b006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.831+0000 7f91ae800640 1 -- 172.21.15.150:0/4226720328 >> v2:172.21.15.187:3300/0 conn(0x7f91b00cee00 msgr2=0x7f91b006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:10.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.831+0000 7f91ae800640 1 --2- 172.21.15.150:0/4226720328 >> v2:172.21.15.187:3300/0 conn(0x7f91b00cee00 0x7f91b006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.831+0000 7f91ae800640 1 -- 172.21.15.150:0/4226720328 >> v1:172.21.15.150:6789/0 conn(0x7f91b0071f50 legacy=0x7f91b015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.831+0000 7f91ae800640 1 -- 172.21.15.150:0/4226720328 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b015d8d0 con 0x7f91b015d2f0 2024-04-15T15:26:10.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.831+0000 7f91ad400640 1 -- 172.21.15.150:0/4226720328 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3783948841 0 0) 0x7f91b015dbb0 con 0x7f91b0071f50 2024-04-15T15:26:10.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.832+0000 7f91ae800640 1 --2- 172.21.15.150:0/4226720328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00723b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f91b0070360 tx=0x7f91a802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03567160333746b server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.832+0000 7f91ad400640 1 -- 172.21.15.150:0/4226720328 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a802cb20 con 0x7f91b015d2f0 2024-04-15T15:26:10.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.832+0000 7f91ad400640 1 -- 172.21.15.150:0/4226720328 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a802ccc0 con 0x7f91b015d2f0 2024-04-15T15:26:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.832+0000 7f91ad400640 1 -- 172.21.15.150:0/4226720328 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8034610 con 0x7f91b015d2f0 2024-04-15T15:26:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.833+0000 7f91af200640 1 -- 172.21.15.150:0/4226720328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 msgr2=0x7f91b00723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.833+0000 7f91af200640 1 --2- 172.21.15.150:0/4226720328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00723b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f91b0070360 tx=0x7f91a802bf80 comp rx=0 tx=0).stop 2024-04-15T15:26:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.833+0000 7f91af200640 1 -- 172.21.15.150:0/4226720328 shutdown_connections 2024-04-15T15:26:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.833+0000 7f91af200640 1 --2- 172.21.15.150:0/4226720328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00723b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.833+0000 7f91af200640 1 --2- 172.21.15.150:0/4226720328 >> v2:172.21.15.187:3300/0 conn(0x7f91b00cee00 0x7f91b006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.833+0000 7f91af200640 1 -- 172.21.15.150:0/4226720328 >> 172.21.15.150:0/4226720328 conn(0x7f91b006b930 msgr2=0x7f91b006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.833+0000 7f91af200640 1 -- 172.21.15.150:0/4226720328 shutdown_connections 2024-04-15T15:26:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.833+0000 7f91af200640 1 -- 172.21.15.150:0/4226720328 wait complete. 2024-04-15T15:26:10.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.835+0000 7f91af200640 1 Processor -- start 2024-04-15T15:26:10.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.835+0000 7f91af200640 1 -- start start 2024-04-15T15:26:10.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91af200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91b0071f50 0x7f91b00ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91af200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91b00cee00 0x7f91b00abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91af200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91af200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91b006e3a0 con 0x7f91b015d2f0 2024-04-15T15:26:10.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91af200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91b006e0c0 con 0x7f91b0071f50 2024-04-15T15:26:10.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91af200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f91b015dda0 con 0x7f91b00cee00 2024-04-15T15:26:10.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91ae800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91ae800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00ac250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46470/0 (socket says 172.21.15.150:46470) 2024-04-15T15:26:10.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91ae800640 1 -- 172.21.15.150:0/524662970 learned_addr learned my addr 172.21.15.150:0/524662970 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:10.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91ae800640 1 -- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91b00cee00 msgr2=0x7f91b00abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91ae800640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91b00cee00 0x7f91b00abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91ade00640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91b0071f50 0x7f91b00ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.836+0000 7f91ae800640 1 -- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91b0071f50 msgr2=0x7f91b00ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:10.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.837+0000 7f91ae800640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91b0071f50 0x7f91b00ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:10.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.837+0000 7f91ae800640 1 -- 172.21.15.150:0/524662970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b01f25a0 con 0x7f91b015d2f0 2024-04-15T15:26:10.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.837+0000 7f91ae800640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00ac250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009f40 tx=0x7f91a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.837+0000 7f91a6a00640 1 -- 172.21.15.150:0/524662970 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8043020 con 0x7f91b015d2f0 2024-04-15T15:26:10.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.837+0000 7f91a6a00640 1 -- 172.21.15.150:0/524662970 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a803e450 con 0x7f91b015d2f0 2024-04-15T15:26:10.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.837+0000 7f91a6a00640 1 -- 172.21.15.150:0/524662970 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8009510 con 0x7f91b015d2f0 2024-04-15T15:26:10.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.837+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b00ac880 con 0x7f91b015d2f0 2024-04-15T15:26:10.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.837+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b00ace40 con 0x7f91b015d2f0 2024-04-15T15:26:10.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.839+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b00726e0 con 0x7f91b015d2f0 2024-04-15T15:26:10.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.839+0000 7f91a6a00640 1 -- 172.21.15.150:0/524662970 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f91a8046020 con 0x7f91b015d2f0 2024-04-15T15:26:10.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.840+0000 7f91a6a00640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f918c06c980 0x7f918c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:10.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.841+0000 7f91a6a00640 1 -- 172.21.15.150:0/524662970 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f91a80ae1b0 con 0x7f91b015d2f0 2024-04-15T15:26:10.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.841+0000 7f91ade00640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f918c06c980 0x7f918c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:10.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.842+0000 7f91ade00640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f918c06c980 0x7f918c06ee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:10.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:10.845+0000 7f91a6a00640 1 -- 172.21.15.150:0/524662970 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91b00726e0 con 0x7f91b015d2f0 2024-04-15T15:26:11.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.043+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91b00755c0 con 0x7f918c06c980 2024-04-15T15:26:11.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.045+0000 7f91a6a00640 1 -- 172.21.15.150:0/524662970 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13353 (secure 0 0 0) 0x7f91b00755c0 con 0x7f918c06c980 2024-04-15T15:26:11.047 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:26:11.048 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:26:11.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.047+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f918c06c980 msgr2=0x7f918c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:11.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.047+0000 7f91af200640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f918c06c980 0x7f918c06ee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0008040 comp rx=0 tx=0).stop 2024-04-15T15:26:11.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 msgr2=0x7f91b00ac250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:11.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00ac250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009f40 tx=0x7f91a8004a40 comp rx=0 tx=0).stop 2024-04-15T15:26:11.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 shutdown_connections 2024-04-15T15:26:11.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b015d2f0 0x7f91b00ac250 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:11.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f918c06c980 0x7f918c06ee40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:11.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91b00cee00 0x7f91b00abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:11.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 --2- 172.21.15.150:0/524662970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91b0071f50 0x7f91b00ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:11.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 >> 172.21.15.150:0/524662970 conn(0x7f91b006b930 msgr2=0x7f91b015a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:11.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 shutdown_connections 2024-04-15T15:26:11.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:11.048+0000 7f91af200640 1 -- 172.21.15.150:0/524662970 wait complete. 2024-04-15T15:26:11.069 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-04-15T15:26:09.263667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229148,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745572,"statfs":{"total":773094113280,"available":772859465728,"internally_reserved":0,"allocated":4759552,"data_stored":2591088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.819099"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":203,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-15T15:25:58.966944+0000","last_change":"2024-04-15T15:23:45.287148+0000","last_active":"2024-04-15T15:25:58.966944+0000","last_peered":"2024-04-15T15:25:58.966944+0000","last_clean":"2024-04-15T15:25:58.966944+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:25:58.966944+0000","last_undegraded":"2024-04-15T15:25:58.966944+0000","last_fullsized":"2024-04-15T15:25:58.966944+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-15T15:23:44.109239+0000","last_clean_scrub_stamp":"2024-04-15T15:23:44.109239+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T17:21:53.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1040384,"data_stored":766662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":766662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:26:11.072 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-15T15:23:44.109239+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=15, tm_hour=15, tm_min=23, tm_sec=44, tm_wday=0, tm_yday=106, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=15, tm_hour=15, tm_min=26, tm_sec=3, tm_wday=0, tm_yday=106, tm_isdst=0) 2024-04-15T15:26:11.072 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-15T15:26:17.095 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:26:17.103 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:26:31.073 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-15T15:26:31.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.291+0000 7fcaefe00640 1 Processor -- start 2024-04-15T15:26:31.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.291+0000 7fcaefe00640 1 -- start start 2024-04-15T15:26:31.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.293+0000 7fcaefe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcaf00cee00 0x7fcaf006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.293+0000 7fcaefe00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf015dbb0 con 0x7fcaf0071f50 2024-04-15T15:26:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.293+0000 7fcaefe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf015d8d0 con 0x7fcaf015d2f0 2024-04-15T15:26:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.293+0000 7fcaefe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf006f4f0 con 0x7fcaf00cee00 2024-04-15T15:26:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.293+0000 7fcae6000640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fcaf0071f50 0x7fcaf015cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:36898/0 (socket says 172.21.15.150:36898) 2024-04-15T15:26:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.293+0000 7fcae6000640 1 -- 172.21.15.150:0/4036508457 learned_addr learned my addr 172.21.15.150:0/4036508457 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-15T15:26:31.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaeea00640 1 --2- 172.21.15.150:0/4036508457 >> v2:172.21.15.187:3300/0 conn(0x7fcaf00cee00 0x7fcaf006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 <== mon.0 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3277875837 0 0) 0x7fcaf015dbb0 con 0x7fcaf0071f50 2024-04-15T15:26:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcad4003520 con 0x7fcaf0071f50 2024-04-15T15:26:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 <== mon.0 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4026367356 0 0) 0x7fcad4003520 con 0x7fcaf0071f50 2024-04-15T15:26:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 >> v2:172.21.15.187:3300/0 conn(0x7fcaf00cee00 msgr2=0x7fcaf006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 --2- 172.21.15.150:0/4036508457 >> v2:172.21.15.187:3300/0 conn(0x7fcaf00cee00 0x7fcaf006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 >> v1:172.21.15.187:6789/0 conn(0x7fcaf015d2f0 legacy=0x7fcaf015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf006dde0 con 0x7fcaf0071f50 2024-04-15T15:26:31.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 <== mon.0 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1276118705 0 0) 0x7fcad8001670 con 0x7fcaf0071f50 2024-04-15T15:26:31.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 >> v1:172.21.15.150:6789/0 conn(0x7fcaf0071f50 legacy=0x7fcaf015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad40039b0 0x7fcad4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4004800 0x7fcad4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcad40051c0 0x7fcad400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcad400c0c0 con 0x7fcad4004800 2024-04-15T15:26:31.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcad400c270 con 0x7fcad40039b0 2024-04-15T15:26:31.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.294+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcad400c470 con 0x7fcad40051c0 2024-04-15T15:26:31.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcae6000640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4004800 0x7fcad4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcaeea00640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad40039b0 0x7fcad4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcaef400640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcad40051c0 0x7fcad400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcae6000640 1 -- 172.21.15.150:0/4036508457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcad40051c0 msgr2=0x7fcad400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcae6000640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcad40051c0 0x7fcad400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcae6000640 1 -- 172.21.15.150:0/4036508457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad40039b0 msgr2=0x7fcad4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcae6000640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad40039b0 0x7fcad4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcae6000640 1 -- 172.21.15.150:0/4036508457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad4003520 con 0x7fcad4004800 2024-04-15T15:26:31.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.295+0000 7fcaef400640 1 -- 172.21.15.150:0/4036508457 reap_dead start 2024-04-15T15:26:31.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.296+0000 7fcae6000640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4004800 0x7fcad4004c80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcad800c210 tx=0x7fcad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f29fc441e421cd server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:31.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.296+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8007d30 con 0x7fcad4004800 2024-04-15T15:26:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad8030070 con 0x7fcad4004800 2024-04-15T15:26:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaee000640 1 -- 172.21.15.150:0/4036508457 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8005d70 con 0x7fcad4004800 2024-04-15T15:26:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaefe00640 1 -- 172.21.15.150:0/4036508457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4004800 msgr2=0x7fcad4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaefe00640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4004800 0x7fcad4004c80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcad800c210 tx=0x7fcad4005830 comp rx=0 tx=0).stop 2024-04-15T15:26:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaefe00640 1 -- 172.21.15.150:0/4036508457 shutdown_connections 2024-04-15T15:26:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaefe00640 1 --2- 172.21.15.150:0/4036508457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcad4004800 0x7fcad4004c80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaefe00640 1 -- 172.21.15.150:0/4036508457 >> 172.21.15.150:0/4036508457 conn(0x7fcaf006b930 msgr2=0x7fcaf006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaefe00640 1 -- 172.21.15.150:0/4036508457 shutdown_connections 2024-04-15T15:26:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.297+0000 7fcaefe00640 1 -- 172.21.15.150:0/4036508457 wait complete. 2024-04-15T15:26:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.299+0000 7fcaefe00640 1 Processor -- start 2024-04-15T15:26:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.299+0000 7fcaefe00640 1 -- start start 2024-04-15T15:26:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.300+0000 7fcaefe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad4004800 0x7fcaf0150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.300+0000 7fcaefe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf015b850 0x7fcaf015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.300+0000 7fcaefe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf00aeba0 0x7fcaf00af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.300+0000 7fcaefe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf006e8c0 con 0x7fcaf00aeba0 2024-04-15T15:26:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.300+0000 7fcaefe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf006e3a0 con 0x7fcad4004800 2024-04-15T15:26:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.300+0000 7fcaefe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf006ea40 con 0x7fcaf015b850 2024-04-15T15:26:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf00aeba0 0x7fcaf00af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf00aeba0 0x7fcaf00af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41322/0 (socket says 172.21.15.150:41322) 2024-04-15T15:26:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 -- 172.21.15.150:0/4043288404 learned_addr learned my addr 172.21.15.150:0/4043288404 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcae6000640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf015b850 0x7fcaf015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 -- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf015b850 msgr2=0x7fcaf015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf015b850 0x7fcaf015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 -- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad4004800 msgr2=0x7fcaf0150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaeea00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad4004800 0x7fcaf0150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad4004800 0x7fcaf0150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 -- 172.21.15.150:0/4043288404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf00ada50 con 0x7fcaf00aeba0 2024-04-15T15:26:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcae6000640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf015b850 0x7fcaf015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaeea00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad4004800 0x7fcaf0150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.301+0000 7fcaef400640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf00aeba0 0x7fcaf00af020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0161270 tx=0x7fcae800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.302+0000 7fcae7400640 1 -- 172.21.15.150:0/4043288404 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8016020 con 0x7fcaf00aeba0 2024-04-15T15:26:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.302+0000 7fcae7400640 1 -- 172.21.15.150:0/4043288404 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae8012800 con 0x7fcaf00aeba0 2024-04-15T15:26:31.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.302+0000 7fcae7400640 1 -- 172.21.15.150:0/4043288404 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8012bf0 con 0x7fcaf00aeba0 2024-04-15T15:26:31.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.302+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf00add30 con 0x7fcaf00aeba0 2024-04-15T15:26:31.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.302+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf00ae2f0 con 0x7fcaf00aeba0 2024-04-15T15:26:31.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.303+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf00755c0 con 0x7fcaf00aeba0 2024-04-15T15:26:31.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.304+0000 7fcae7400640 1 -- 172.21.15.150:0/4043288404 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcae8019020 con 0x7fcaf00aeba0 2024-04-15T15:26:31.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.305+0000 7fcae7400640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcacc06c980 0x7fcacc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.305+0000 7fcae7400640 1 -- 172.21.15.150:0/4043288404 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fcae808b7e0 con 0x7fcaf00aeba0 2024-04-15T15:26:31.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.306+0000 7fcaeea00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcacc06c980 0x7fcacc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.307+0000 7fcaeea00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcacc06c980 0x7fcacc06ee40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0159010 tx=0x7fcae000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:31.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.310+0000 7fcae7400640 1 -- 172.21.15.150:0/4043288404 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcae8020020 con 0x7fcaf00aeba0 2024-04-15T15:26:31.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.506+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 --> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcaf015c210 con 0x7fcacc06c980 2024-04-15T15:26:31.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.509+0000 7fcae7400640 1 -- 172.21.15.150:0/4043288404 <== mgr.4107 v2:172.21.15.187:6832/105751222 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7fcaf015c210 con 0x7fcacc06c980 2024-04-15T15:26:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:26:31.511 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-15T15:26:31.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcacc06c980 msgr2=0x7fcacc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcacc06c980 0x7fcacc06ee40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0159010 tx=0x7fcae000b040 comp rx=0 tx=0).stop 2024-04-15T15:26:31.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf00aeba0 msgr2=0x7fcaf00af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf00aeba0 0x7fcaf00af020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0161270 tx=0x7fcae800d890 comp rx=0 tx=0).stop 2024-04-15T15:26:31.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 shutdown_connections 2024-04-15T15:26:31.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf00aeba0 0x7fcaf00af020 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7fcacc06c980 0x7fcacc06ee40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf015b850 0x7fcaf015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 --2- 172.21.15.150:0/4043288404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcad4004800 0x7fcaf0150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 >> 172.21.15.150:0/4043288404 conn(0x7fcaf006b930 msgr2=0x7fcaf00ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:31.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 shutdown_connections 2024-04-15T15:26:31.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.511+0000 7fcaefe00640 1 -- 172.21.15.150:0/4043288404 wait complete. 2024-04-15T15:26:31.531 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-04-15T15:26:31.270067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229140,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745580,"statfs":{"total":773094113280,"available":772859473920,"internally_reserved":0,"allocated":4759552,"data_stored":2591088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003574"},"pg_stats":[{"pgid":"1.0","version":"39'119","reported_seq":214,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-15T15:26:11.565563+0000","last_change":"2024-04-15T15:26:11.565663+0000","last_active":"2024-04-15T15:26:11.565563+0000","last_peered":"2024-04-15T15:26:11.565563+0000","last_clean":"2024-04-15T15:26:11.565563+0000","last_became_active":"2024-04-15T15:23:45.285007+0000","last_became_peered":"2024-04-15T15:23:45.285007+0000","last_unstale":"2024-04-15T15:26:11.565563+0000","last_undegraded":"2024-04-15T15:26:11.565563+0000","last_fullsized":"2024-04-15T15:26:11.565563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'119","last_scrub_stamp":"2024-04-15T15:26:11.565376+0000","last_deep_scrub":"39'119","last_deep_scrub_stamp":"2024-04-15T15:26:11.565376+0000","last_clean_scrub_stamp":"2024-04-15T15:26:11.565376+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T16:13:57.999498+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509475,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1040384,"data_stored":766662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31620,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1040384,"data_stored":766662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":176294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-15T15:26:31.532 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-15T15:26:31.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.729+0000 7f6c1a200640 1 Processor -- start 2024-04-15T15:26:31.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.729+0000 7f6c1a200640 1 -- start start 2024-04-15T15:26:31.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.731+0000 7f6c1a200640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6c1406f920 0x7f6c141626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.731+0000 7f6c1a200640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c140aebd0 con 0x7f6c14147c90 2024-04-15T15:26:31.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.731+0000 7f6c1a200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c140aee50 con 0x7f6c141478b0 2024-04-15T15:26:31.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.731+0000 7f6c1a200640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f6c140af110 con 0x7f6c1406f920 2024-04-15T15:26:31.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.731+0000 7f6c19800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6c1406f920 0x7f6c141626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.732+0000 7f6c19800640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6c1406f920 0x7f6c141626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41334/0 (socket says 172.21.15.150:41334) 2024-04-15T15:26:31.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.732+0000 7f6c19800640 1 -- 172.21.15.150:0/4005811146 learned_addr learned my addr 172.21.15.150:0/4005811146 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:31.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.732+0000 7f6c13e00640 1 --1- 172.21.15.150:0/4005811146 >> v1:172.21.15.150:6789/0 conn(0x7f6c14147c90 0x7f6c1406f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:36908/0 (socket says 172.21.15.150:36908) 2024-04-15T15:26:31.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.732+0000 7f6c19800640 1 -- 172.21.15.150:0/4005811146 >> v1:172.21.15.187:6789/0 conn(0x7f6c141478b0 legacy=0x7f6c140afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:31.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.732+0000 7f6c19800640 1 -- 172.21.15.150:0/4005811146 >> v1:172.21.15.150:6789/0 conn(0x7f6c14147c90 legacy=0x7f6c1406f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.732+0000 7f6c19800640 1 -- 172.21.15.150:0/4005811146 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c140af3f0 con 0x7f6c1406f920 2024-04-15T15:26:31.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.733+0000 7f6c19800640 1 --2- 172.21.15.150:0/4005811146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1406f920 0x7f6c141626c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009e50 tx=0x7f6c0c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b193e4a602f1f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:31.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.734+0000 7f6c13400640 1 -- 172.21.15.150:0/4005811146 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c02cb20 con 0x7f6c1406f920 2024-04-15T15:26:31.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.734+0000 7f6c13400640 1 -- 172.21.15.150:0/4005811146 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0c02ccc0 con 0x7f6c1406f920 2024-04-15T15:26:31.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.734+0000 7f6c13400640 1 -- 172.21.15.150:0/4005811146 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c034610 con 0x7f6c1406f920 2024-04-15T15:26:31.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.734+0000 7f6c1a200640 1 -- 172.21.15.150:0/4005811146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1406f920 msgr2=0x7f6c141626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.734+0000 7f6c1a200640 1 --2- 172.21.15.150:0/4005811146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1406f920 0x7f6c141626c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009e50 tx=0x7f6c0c02bf80 comp rx=0 tx=0).stop 2024-04-15T15:26:31.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.734+0000 7f6c1a200640 1 -- 172.21.15.150:0/4005811146 shutdown_connections 2024-04-15T15:26:31.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.734+0000 7f6c1a200640 1 --2- 172.21.15.150:0/4005811146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1406f920 0x7f6c141626c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.734+0000 7f6c1a200640 1 -- 172.21.15.150:0/4005811146 >> 172.21.15.150:0/4005811146 conn(0x7f6c1406b930 msgr2=0x7f6c1406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:31.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.735+0000 7f6c1a200640 1 -- 172.21.15.150:0/4005811146 shutdown_connections 2024-04-15T15:26:31.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.735+0000 7f6c1a200640 1 -- 172.21.15.150:0/4005811146 wait complete. 2024-04-15T15:26:31.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.736+0000 7f6c1a200640 1 Processor -- start 2024-04-15T15:26:31.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.736+0000 7f6c1a200640 1 -- start start 2024-04-15T15:26:31.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.737+0000 7f6c1a200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1406f920 0x7f6c141407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.737+0000 7f6c1a200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c141478b0 0x7f6c141f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.737+0000 7f6c1a200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c14147c90 0x7f6c140b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.737+0000 7f6c1a200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14162d80 con 0x7f6c141478b0 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.737+0000 7f6c1a200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14162c00 con 0x7f6c14147c90 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.737+0000 7f6c1a200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6c14162f00 con 0x7f6c1406f920 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c141478b0 0x7f6c141f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c141478b0 0x7f6c141f4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41350/0 (socket says 172.21.15.150:41350) 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 -- 172.21.15.150:0/770880761 learned_addr learned my addr 172.21.15.150:0/770880761 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:31.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c18e00640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1406f920 0x7f6c141407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 -- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1406f920 msgr2=0x7f6c141407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1406f920 0x7f6c141407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 -- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c14147c90 msgr2=0x7f6c140b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c14147c90 0x7f6c140b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 -- 172.21.15.150:0/770880761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c141f5980 con 0x7f6c141478b0 2024-04-15T15:26:31.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c18e00640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1406f920 0x7f6c141407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:31.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.738+0000 7f6c13e00640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c141478b0 0x7f6c141f4780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000c3c0 tx=0x7f6c0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:31.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.739+0000 7f6c10c00640 1 -- 172.21.15.150:0/770880761 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c000074e0 con 0x7f6c141478b0 2024-04-15T15:26:31.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.739+0000 7f6c10c00640 1 -- 172.21.15.150:0/770880761 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c00007680 con 0x7f6c141478b0 2024-04-15T15:26:31.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.739+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c141f53c0 con 0x7f6c141478b0 2024-04-15T15:26:31.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.739+0000 7f6c10c00640 1 -- 172.21.15.150:0/770880761 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00004a00 con 0x7f6c141478b0 2024-04-15T15:26:31.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.739+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c141f5d60 con 0x7f6c141478b0 2024-04-15T15:26:31.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.740+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c141f5f60 con 0x7f6c141478b0 2024-04-15T15:26:31.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.741+0000 7f6c10c00640 1 -- 172.21.15.150:0/770880761 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6c0001a020 con 0x7f6c141478b0 2024-04-15T15:26:31.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.742+0000 7f6c10c00640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6bf406c980 0x7f6bf406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:31.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.742+0000 7f6c10c00640 1 -- 172.21.15.150:0/770880761 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f6c0008b1a0 con 0x7f6c141478b0 2024-04-15T15:26:31.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.743+0000 7f6c18e00640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6bf406c980 0x7f6bf406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:31.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.744+0000 7f6c18e00640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6bf406c980 0x7f6bf406ee40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c140cf8d0 tx=0x7f6c08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:31.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.747+0000 7f6c10c00640 1 -- 172.21.15.150:0/770880761 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c141f5f60 con 0x7f6c141478b0 2024-04-15T15:26:31.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.949+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f6c140755c0 con 0x7f6c141478b0 2024-04-15T15:26:31.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.954+0000 7f6c10c00640 1 -- 172.21.15.150:0/770880761 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f6c140aebd0 con 0x7f6c141478b0 2024-04-15T15:26:31.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.956+0000 7f6c10c00640 1 -- 172.21.15.150:0/770880761 <== mon.0 v2:172.21.15.150:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6c00059780 con 0x7f6c141478b0 2024-04-15T15:26:31.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6bf406c980 msgr2=0x7f6bf406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6bf406c980 0x7f6bf406ee40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c140cf8d0 tx=0x7f6c08009040 comp rx=0 tx=0).stop 2024-04-15T15:26:31.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c141478b0 msgr2=0x7f6c141f4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:31.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c141478b0 0x7f6c141f4780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000c3c0 tx=0x7f6c0000c880 comp rx=0 tx=0).stop 2024-04-15T15:26:31.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 shutdown_connections 2024-04-15T15:26:31.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c14147c90 0x7f6c140b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c141478b0 0x7f6c141f4780 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f6bf406c980 0x7f6bf406ee40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 --2- 172.21.15.150:0/770880761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c1406f920 0x7f6c141407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:31.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 >> 172.21.15.150:0/770880761 conn(0x7f6c1406b930 msgr2=0x7f6c140b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:31.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 shutdown_connections 2024-04-15T15:26:31.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:31.958+0000 7f6c1a200640 1 -- 172.21.15.150:0/770880761 wait complete. 2024-04-15T15:26:31.978 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-15T15:26:31.979 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-15T15:26:32.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.186+0000 7f1ce4400640 1 Processor -- start 2024-04-15T15:26:32.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.186+0000 7f1ce4400640 1 -- start start 2024-04-15T15:26:32.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce4400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0728f0 0x7f1cdc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:32.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce4400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1cdc0affc0 0x7f1cdc1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:32.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce4400640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cdc1485d0 con 0x7f1cdc1481f0 2024-04-15T15:26:32.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce4400640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 -- 0x7f1cdc15ffc0 con 0x7f1cdc0affc0 2024-04-15T15:26:32.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce4400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1cdc0aef20 con 0x7f1cdc0728f0 2024-04-15T15:26:32.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce2400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1cdc0affc0 0x7f1cdc1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:32.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce2400640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1cdc0affc0 0x7f1cdc1601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41356/0 (socket says 172.21.15.150:41356) 2024-04-15T15:26:32.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce2400640 1 -- 172.21.15.150:0/834909029 learned_addr learned my addr 172.21.15.150:0/834909029 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:32.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.188+0000 7f1ce1a00640 1 --1- 172.21.15.150:0/834909029 >> v1:172.21.15.150:6789/0 conn(0x7f1cdc1481f0 0x7f1cdc0721e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:36920/0 (socket says 172.21.15.150:36920) 2024-04-15T15:26:32.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.189+0000 7f1ce1000640 1 --2- 172.21.15.150:0/834909029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0728f0 0x7f1cdc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:32.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.189+0000 7f1ce2400640 1 -- 172.21.15.150:0/834909029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0728f0 msgr2=0x7f1cdc15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-15T15:26:32.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.189+0000 7f1ce2400640 1 --2- 172.21.15.150:0/834909029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0728f0 0x7f1cdc15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.189+0000 7f1ce2400640 1 -- 172.21.15.150:0/834909029 >> v1:172.21.15.150:6789/0 conn(0x7f1cdc1481f0 legacy=0x7f1cdc0721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:32.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.189+0000 7f1ce2400640 1 -- 172.21.15.150:0/834909029 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc0af200 con 0x7f1cdc0affc0 2024-04-15T15:26:32.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.189+0000 7f1ce1000640 1 --2- 172.21.15.150:0/834909029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0728f0 0x7f1cdc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:32.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.190+0000 7f1ce2400640 1 --2- 172.21.15.150:0/834909029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0affc0 0x7f1cdc1601a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009e50 tx=0x7f1cd402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9645c51024048c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:32.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.190+0000 7f1cdbe00640 1 -- 172.21.15.150:0/834909029 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd402cb00 con 0x7f1cdc0affc0 2024-04-15T15:26:32.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.190+0000 7f1cdbe00640 1 -- 172.21.15.150:0/834909029 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1cd402cca0 con 0x7f1cdc0affc0 2024-04-15T15:26:32.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.190+0000 7f1cdbe00640 1 -- 172.21.15.150:0/834909029 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4034630 con 0x7f1cdc0affc0 2024-04-15T15:26:32.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.191+0000 7f1ce4400640 1 -- 172.21.15.150:0/834909029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0affc0 msgr2=0x7f1cdc1601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:32.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.191+0000 7f1ce4400640 1 --2- 172.21.15.150:0/834909029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0affc0 0x7f1cdc1601a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009e50 tx=0x7f1cd402bf60 comp rx=0 tx=0).stop 2024-04-15T15:26:32.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.191+0000 7f1ce4400640 1 -- 172.21.15.150:0/834909029 shutdown_connections 2024-04-15T15:26:32.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.191+0000 7f1ce4400640 1 --2- 172.21.15.150:0/834909029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0728f0 0x7f1cdc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.191+0000 7f1ce4400640 1 --2- 172.21.15.150:0/834909029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0affc0 0x7f1cdc1601a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.191+0000 7f1ce4400640 1 -- 172.21.15.150:0/834909029 >> 172.21.15.150:0/834909029 conn(0x7f1cdc0b27b0 msgr2=0x7f1cdc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:32.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.191+0000 7f1ce4400640 1 -- 172.21.15.150:0/834909029 shutdown_connections 2024-04-15T15:26:32.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.191+0000 7f1ce4400640 1 -- 172.21.15.150:0/834909029 wait complete. 2024-04-15T15:26:32.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.193+0000 7f1ce4400640 1 Processor -- start 2024-04-15T15:26:32.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.194+0000 7f1ce4400640 1 -- start start 2024-04-15T15:26:32.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce4400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0728f0 0x7f1cdc15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:32.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce4400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0affc0 0x7f1cdc06f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:32.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce4400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cdc1481f0 0x7f1cdc13e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:32.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce4400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc0afaa0 con 0x7f1cdc0728f0 2024-04-15T15:26:32.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce4400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc0af7c0 con 0x7f1cdc1481f0 2024-04-15T15:26:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce4400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1cdc0b5560 con 0x7f1cdc0affc0 2024-04-15T15:26:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce1a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0728f0 0x7f1cdc15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce1a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0728f0 0x7f1cdc15d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41370/0 (socket says 172.21.15.150:41370) 2024-04-15T15:26:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce1a00640 1 -- 172.21.15.150:0/3423671715 learned_addr learned my addr 172.21.15.150:0/3423671715 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-15T15:26:32.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce1a00640 1 -- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0affc0 msgr2=0x7f1cdc06f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.195+0000 7f1ce1000640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0affc0 0x7f1cdc06f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce1a00640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0affc0 0x7f1cdc06f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce2400640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cdc1481f0 0x7f1cdc13e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce1a00640 1 -- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cdc1481f0 msgr2=0x7f1cdc13e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:32.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce1a00640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cdc1481f0 0x7f1cdc13e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce1a00640 1 -- 172.21.15.150:0/3423671715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc0cd450 con 0x7f1cdc0728f0 2024-04-15T15:26:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce1000640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0affc0 0x7f1cdc06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-15T15:26:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce1a00640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0728f0 0x7f1cdc15d9d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000c3c0 tx=0x7f1cd000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1cd9600640 1 -- 172.21.15.150:0/3423671715 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00074e0 con 0x7f1cdc0728f0 2024-04-15T15:26:32.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1cd9600640 1 -- 172.21.15.150:0/3423671715 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1cd0007680 con 0x7f1cdc0728f0 2024-04-15T15:26:32.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1cd9600640 1 -- 172.21.15.150:0/3423671715 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0004a20 con 0x7f1cdc0728f0 2024-04-15T15:26:32.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cdc0cd730 con 0x7f1cdc0728f0 2024-04-15T15:26:32.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.196+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cdc0cdcf0 con 0x7f1cdc0728f0 2024-04-15T15:26:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.197+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cdc068c80 con 0x7f1cdc0728f0 2024-04-15T15:26:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.199+0000 7f1cd9600640 1 -- 172.21.15.150:0/3423671715 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1cd001a020 con 0x7f1cdc0728f0 2024-04-15T15:26:32.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.199+0000 7f1cd9600640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1cbc06c9b0 0x7f1cbc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-15T15:26:32.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.200+0000 7f1cd9600640 1 -- 172.21.15.150:0/3423671715 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f1cd008b270 con 0x7f1cdc0728f0 2024-04-15T15:26:32.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.200+0000 7f1ce1000640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1cbc06c9b0 0x7f1cbc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-15T15:26:32.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.201+0000 7f1ce1000640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1cbc06c9b0 0x7f1cbc06ee70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8006fd0 tx=0x7f1cc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-15T15:26:32.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.204+0000 7f1cd9600640 1 -- 172.21.15.150:0/3423671715 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1cd0010300 con 0x7f1cdc0728f0 2024-04-15T15:26:32.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.445+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cdc15fb90 con 0x7f1cdc0728f0 2024-04-15T15:26:32.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.446+0000 7f1cd9600640 1 -- 172.21.15.150:0/3423671715 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+3881 (secure 0 0 0) 0x7f1cd00597c0 con 0x7f1cdc0728f0 2024-04-15T15:26:32.448 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-15T15:26:32.448 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3733235669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3733235669},{"type":"v1","addr":"172.21.15.187:6836","nonce":3733235669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1872688632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":1872688632},{"type":"v1","addr":"172.21.15.187:6837","nonce":1872688632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2755110957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2755110957},{"type":"v1","addr":"172.21.15.150:6834","nonce":2755110957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2436628798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2436628798},{"type":"v1","addr":"172.21.15.150:6835","nonce":2436628798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[]} 2024-04-15T15:26:32.448 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 24 2024-04-15T15:26:32.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1cbc06c9b0 msgr2=0x7f1cbc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:32.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1cbc06c9b0 0x7f1cbc06ee70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8006fd0 tx=0x7f1cc8008040 comp rx=0 tx=0).stop 2024-04-15T15:26:32.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0728f0 msgr2=0x7f1cdc15d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-15T15:26:32.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0728f0 0x7f1cdc15d9d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000c3c0 tx=0x7f1cd000c880 comp rx=0 tx=0).stop 2024-04-15T15:26:32.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 shutdown_connections 2024-04-15T15:26:32.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cdc1481f0 0x7f1cdc13e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:6832/105751222,v1:172.21.15.187:6833/105751222] conn(0x7f1cbc06c9b0 0x7f1cbc06ee70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cdc0affc0 0x7f1cdc06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 --2- 172.21.15.150:0/3423671715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cdc0728f0 0x7f1cdc15d9d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-15T15:26:32.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 >> 172.21.15.150:0/3423671715 conn(0x7f1cdc0b27b0 msgr2=0x7f1cdc0b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-15T15:26:32.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 shutdown_connections 2024-04-15T15:26:32.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-15T15:26:32.449+0000 7f1ce4400640 1 -- 172.21.15.150:0/3423671715 wait complete. 2024-04-15T15:26:32.470 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-15T15:26:32.471 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-15T15:26:32.471 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-15T15:26:32.471 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.489 INFO:tasks.ceph.mds.a:Stopped 2024-04-15T15:26:32.490 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-15T15:26:32.490 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.558 INFO:tasks.ceph.mds.c:Stopped 2024-04-15T15:26:32.559 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-15T15:26:32.559 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.578 INFO:tasks.ceph.mds.b:Stopped 2024-04-15T15:26:32.578 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-15T15:26:32.578 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.639 INFO:tasks.ceph.mds.d:Stopped 2024-04-15T15:26:32.639 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-15T15:26:32.639 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-15T15:26:32.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.698 INFO:tasks.ceph.osd.0:Stopped 2024-04-15T15:26:32.698 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-15T15:26:32.699 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.760 INFO:tasks.ceph.osd.1:Stopped 2024-04-15T15:26:32.761 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-15T15:26:32.761 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.823 INFO:tasks.ceph.osd.2:Stopped 2024-04-15T15:26:32.823 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-15T15:26:32.823 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.894 INFO:tasks.ceph.osd.3:Stopped 2024-04-15T15:26:32.895 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-15T15:26:32.895 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:32.951 INFO:tasks.ceph.osd.4:Stopped 2024-04-15T15:26:32.951 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-15T15:26:32.951 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:33.013 INFO:tasks.ceph.osd.5:Stopped 2024-04-15T15:26:33.013 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-15T15:26:33.013 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:33.071 INFO:tasks.ceph.osd.6:Stopped 2024-04-15T15:26:33.072 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-15T15:26:33.072 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:33.144 INFO:tasks.ceph.osd.7:Stopped 2024-04-15T15:26:33.145 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-15T15:26:33.145 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-15T15:26:33.145 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:34.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-15T15:26:39.247 INFO:tasks.ceph.mgr.y:Stopped 2024-04-15T15:26:39.247 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-15T15:26:39.247 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:39.332 INFO:tasks.ceph.mgr.x:Stopped 2024-04-15T15:26:39.332 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-15T15:26:39.332 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-15T15:26:39.333 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:39.365 INFO:tasks.ceph.mon.a:Stopped 2024-04-15T15:26:39.365 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-15T15:26:39.365 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:39.403 INFO:tasks.ceph.mon.b:Stopped 2024-04-15T15:26:39.404 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-15T15:26:39.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-15T15:26:39.489 INFO:tasks.ceph.mon.c:Stopped 2024-04-15T15:26:39.490 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-15T15:26:39.490 DEBUG:teuthology.orchestra.run.smithi150:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow request' | head -n 1 2024-04-15T15:26:39.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:26:39.599 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-15T15:26:39.857 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:26:39.857 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-15T15:26:39.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-15T15:26:40.033 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:26:40.033 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-15T15:26:40.204 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:26:40.204 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-15T15:26:40.411 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:26:40.411 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-15T15:26:40.602 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:26:40.602 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-15T15:26:40.744 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:26:40.745 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-15T15:26:40.894 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:26:40.895 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-15T15:26:41.034 INFO:tasks.ceph:Archiving mon data... 2024-04-15T15:26:41.035 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/data/mon.a.tgz 2024-04-15T15:26:41.036 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-04-15T15:26:41.095 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.aqU3GqJ6ke 2024-04-15T15:26:41.095 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.aqU3GqJ6ke 2024-04-15T15:26:41.270 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.aqU3GqJ6ke 2024-04-15T15:26:41.416 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.aqU3GqJ6ke is 348KB 2024-04-15T15:26:41.517 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.aqU3GqJ6ke 2024-04-15T15:26:41.575 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/data/mon.b.tgz 2024-04-15T15:26:41.575 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-15T15:26:41.633 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.2OlwAHuDYf 2024-04-15T15:26:41.633 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2OlwAHuDYf 2024-04-15T15:26:41.873 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.2OlwAHuDYf 2024-04-15T15:26:42.026 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.2OlwAHuDYf is 466KB 2024-04-15T15:26:42.134 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.2OlwAHuDYf 2024-04-15T15:26:42.190 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/data/mon.c.tgz 2024-04-15T15:26:42.190 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-15T15:26:42.243 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.894YzWUkgg 2024-04-15T15:26:42.243 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.894YzWUkgg 2024-04-15T15:26:42.430 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.894YzWUkgg 2024-04-15T15:26:42.560 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.894YzWUkgg is 466KB 2024-04-15T15:26:42.665 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.894YzWUkgg 2024-04-15T15:26:42.721 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-15T15:26:42.722 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-15T15:26:42.729 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-15T15:26:42.877 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-15T15:26:42.936 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-15T15:26:42.936 INFO:tasks.ceph:Archiving crash dumps... 2024-04-15T15:26:42.938 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/remote/smithi150/crash 2024-04-15T15:26:42.939 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-15T15:26:43.033 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/remote/smithi187/crash 2024-04-15T15:26:43.034 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-15T15:26:43.132 INFO:tasks.ceph:Shutting down logrotate 2024-04-15T15:26:43.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:26:43.140 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-15T15:26:43.274 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-15T15:26:43.373 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-15T15:26:43.472 INFO:tasks.ceph:Compressing logs... 2024-04-15T15:26:43.473 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-15T15:26:43.480 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-15T15:26:43.566 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46260.log 2024-04-15T15:26:43.567 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47044.log 2024-04-15T15:26:43.568 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42320.log 2024-04-15T15:26:43.568 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46260.log.gz 2024-04-15T15:26:43.569 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48489.log 2024-04-15T15:26:43.569 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-15T15:26:43.569 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47044.log.gz 2024-04-15T15:26:43.570 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2024-04-15T15:26:43.570 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42320.log.gz 2024-04-15T15:26:43.571 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44451.log 2024-04-15T15:26:43.572 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37177.log 2024-04-15T15:26:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48489.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-15T15:26:43.573 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48489.log.gz 2024-04-15T15:26:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42106.log 2024-04-15T15:26:43.573 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2024-04-15T15:26:43.574 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46344.log 2024-04-15T15:26:43.574 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.3.44451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44451.log.gz 2024-04-15T15:26:43.575 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37057.log 2024-04-15T15:26:43.575 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37177.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37177.log.gz 2024-04-15T15:26:43.575 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42861.log 2024-04-15T15:26:43.577 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37257.log 2024-04-15T15:26:43.577 INFO:teuthology.orchestra.run.smithi187.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37057.log.gz 2024-04-15T15:26:43.577 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.42861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42861.log.gz 2024-04-15T15:26:43.577 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37137.log 2024-04-15T15:26:43.578 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43874.log 2024-04-15T15:26:43.578 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42106.log.gz 2024-04-15T15:26:43.578 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53320.log 2024-04-15T15:26:43.578 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46344.log.gz 2024-04-15T15:26:43.578 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-04-15T15:26:43.579 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43874.log.gz 2024-04-15T15:26:43.579 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46050.log 2024-04-15T15:26:43.579 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53320.log.gz 2024-04-15T15:26:43.581 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37257.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37257.log.gz 2024-04-15T15:26:43.581 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-15T15:26:43.581 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37137.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-15T15:26:43.581 INFO:teuthology.orchestra.run.smithi187.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37137.log.gz 2024-04-15T15:26:43.581 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37217.log 2024-04-15T15:26:43.581 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52839.log 2024-04-15T15:26:43.581 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-04-15T15:26:43.582 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45672.log 2024-04-15T15:26:43.582 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46050.log.gz 2024-04-15T15:26:43.583 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52839.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52839.log.gz 2024-04-15T15:26:43.583 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2024-04-15T15:26:43.583 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47487.log 2024-04-15T15:26:43.584 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-15T15:26:43.584 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37217.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-15T15:26:43.584 INFO:teuthology.orchestra.run.smithi187.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37217.log.gz 2024-04-15T15:26:43.584 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37337.log 2024-04-15T15:26:43.584 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-15T15:26:43.585 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37337.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-15T15:26:43.585 INFO:teuthology.orchestra.run.smithi187.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37337.log.gz 2024-04-15T15:26:43.586 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45672.log.gz 2024-04-15T15:26:43.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42236.log 2024-04-15T15:26:43.587 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz 2024-04-15T15:26:43.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42446.log 2024-04-15T15:26:43.587 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47487.log.gz 2024-04-15T15:26:43.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52923.log 2024-04-15T15:26:43.587 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42236.log.gz 2024-04-15T15:26:43.589 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37297.log 2024-04-15T15:26:43.589 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43878.log 2024-04-15T15:26:43.589 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51913.log 2024-04-15T15:26:43.590 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42446.log.gz 2024-04-15T15:26:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47781.log 2024-04-15T15:26:43.590 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52923.log.gz 2024-04-15T15:26:43.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42656.log 2024-04-15T15:26:43.590 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51913.log.gz 2024-04-15T15:26:43.591 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42880.log 2024-04-15T15:26:43.591 INFO:teuthology.orchestra.run.smithi187.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37297.log.gz 2024-04-15T15:26:43.592 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.43878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43878.log.gz 2024-04-15T15:26:43.592 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52881.log 2024-04-15T15:26:43.592 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47781.log.gz 2024-04-15T15:26:43.592 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42836.log 2024-04-15T15:26:43.593 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42656.log.gz 2024-04-15T15:26:43.594 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.42880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42880.log.gz 2024-04-15T15:26:43.594 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37097.log 2024-04-15T15:26:43.594 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-15T15:26:43.595 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2024-04-15T15:26:43.595 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52881.log.gz 2024-04-15T15:26:43.595 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50455.log 2024-04-15T15:26:43.595 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42836.log.gz 2024-04-15T15:26:43.595 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48321.log 2024-04-15T15:26:43.596 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2024-04-15T15:26:43.597 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44235.log 2024-04-15T15:26:43.597 INFO:teuthology.orchestra.run.smithi187.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37097.log.gz 2024-04-15T15:26:43.597 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.0.50455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50455.log.gz 2024-04-15T15:26:43.597 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52461.log 2024-04-15T15:26:43.599 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53762.log 2024-04-15T15:26:43.599 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48321.log.gz 2024-04-15T15:26:43.599 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49432.log 2024-04-15T15:26:43.599 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52461.log.gz 2024-04-15T15:26:43.600 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.1.44235.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-15T15:26:43.600 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44235.log.gz 2024-04-15T15:26:43.600 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44239.log 2024-04-15T15:26:43.601 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48033.log 2024-04-15T15:26:43.601 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53762.log.gz 2024-04-15T15:26:43.601 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.0.49432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49432.log.gz 2024-04-15T15:26:43.601 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45798.log 2024-04-15T15:26:43.603 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.1.44239.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44239.log.gz 2024-04-15T15:26:43.603 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43275.log 2024-04-15T15:26:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51793.log 2024-04-15T15:26:43.604 INFO:teuthology.orchestra.run.smithi150.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48033.log.gz 2024-04-15T15:26:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47991.log 2024-04-15T15:26:43.604 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45798.log.gz 2024-04-15T15:26:43.604 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48968.log 2024-04-15T15:26:43.605 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51793.log.gz 2024-04-15T15:26:43.605 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2024-04-15T15:26:43.606 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47991.log.gz 2024-04-15T15:26:43.607 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-15T15:26:43.607 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-04-15T15:26:43.607 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.3.43275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43275.log.gz 2024-04-15T15:26:43.607 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44447.log 2024-04-15T15:26:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52755.log 2024-04-15T15:26:43.608 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48968.log.gz 2024-04-15T15:26:43.608 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41895.log 2024-04-15T15:26:43.608 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2024-04-15T15:26:43.609 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52419.log 2024-04-15T15:26:43.610 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52755.log.gz 2024-04-15T15:26:43.610 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48884.log 2024-04-15T15:26:43.610 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41895.log.gz 2024-04-15T15:26:43.611 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.3.44447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44447.log.gz 2024-04-15T15:26:43.611 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43874.log 2024-04-15T15:26:43.612 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.43874.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-15T15:26:43.613 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48405.log 2024-04-15T15:26:43.613 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52419.log.gz 2024-04-15T15:26:43.613 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49178.log 2024-04-15T15:26:43.613 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48884.log.gz 2024-04-15T15:26:43.613 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49436.log 2024-04-15T15:26:43.613 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48405.log.gz 2024-04-15T15:26:43.615 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.d.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43874.log.gz 2024-04-15T15:26:43.615 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43279.log 2024-04-15T15:26:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54392.log 2024-04-15T15:26:43.616 INFO:teuthology.orchestra.run.smithi150.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49178.log.gz 2024-04-15T15:26:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.0.49436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49436.log.gz 2024-04-15T15:26:43.616 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52341.log 2024-04-15T15:26:43.618 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-15T15:26:43.618 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50188.log 2024-04-15T15:26:43.618 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54392.log.gz 2024-04-15T15:26:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2024-04-15T15:26:43.619 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52341.log.gz 2024-04-15T15:26:43.619 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46218.log 2024-04-15T15:26:43.619 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50188.log.gz 2024-04-15T15:26:43.621 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8%/var/log/ceph/ceph-client.3.43279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43279.log.gz 2024-04-15T15:26:43.621 INFO:teuthology.orchestra.run.smithi187.stderr: -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-15T15:26:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43174.log 2024-04-15T15:26:43.621 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2024-04-15T15:26:43.621 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2024-04-15T15:26:43.621 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46218.log.gz 2024-04-15T15:26:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48926.log 2024-04-15T15:26:43.623 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43174.log.gz 2024-04-15T15:26:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47403.log 2024-04-15T15:26:43.623 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2024-04-15T15:26:43.624 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52155.log 2024-04-15T15:26:43.624 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48926.log.gz 2024-04-15T15:26:43.625 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53278.log 2024-04-15T15:26:43.625 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47403.log.gz 2024-04-15T15:26:43.626 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54546.log 2024-04-15T15:26:43.626 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52155.log.gz 2024-04-15T15:26:43.627 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43501.log 2024-04-15T15:26:43.627 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53278.log.gz 2024-04-15T15:26:43.628 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42820.log 2024-04-15T15:26:43.628 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54546.log.gz 2024-04-15T15:26:43.629 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37734.log 2024-04-15T15:26:43.629 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43501.log.gz 2024-04-15T15:26:43.630 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49136.log 2024-04-15T15:26:43.631 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42820.log.gz 2024-04-15T15:26:43.631 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43706.log 2024-04-15T15:26:43.631 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37734.log.gz 2024-04-15T15:26:43.632 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50956.log 2024-04-15T15:26:43.632 INFO:teuthology.orchestra.run.smithi150.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49136.log.gz 2024-04-15T15:26:43.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43706.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31326.log 2024-04-15T15:26:43.633 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43706.log.gz 2024-04-15T15:26:43.634 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53049.log 2024-04-15T15:26:43.635 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50956.log.gz 2024-04-15T15:26:43.635 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.tmp-client.admin.31326.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31326.log.gz 2024-04-15T15:26:43.635 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43916.log 2024-04-15T15:26:43.636 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47180.log 2024-04-15T15:26:43.637 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53049.log.gz 2024-04-15T15:26:43.637 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43474.log 2024-04-15T15:26:43.638 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43916.log.gz 2024-04-15T15:26:43.638 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42278.log 2024-04-15T15:26:43.639 INFO:teuthology.orchestra.run.smithi150.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47180.log.gz 2024-04-15T15:26:43.639 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45714.log 2024-04-15T15:26:43.640 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43474.log.gz 2024-04-15T15:26:43.640 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.50751.log 2024-04-15T15:26:43.641 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42278.log.gz 2024-04-15T15:26:43.642 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43748.log 2024-04-15T15:26:43.642 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45714.log.gz 2024-04-15T15:26:43.643 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.2.50751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.50751.log.gz 2024-04-15T15:26:43.643 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44042.log 2024-04-15T15:26:43.643 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47739.log 2024-04-15T15:26:43.644 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43748.log.gz 2024-04-15T15:26:43.645 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53720.log 2024-04-15T15:26:43.645 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44042.log.gz 2024-04-15T15:26:43.646 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44210.log 2024-04-15T15:26:43.646 INFO:teuthology.orchestra.run.smithi150.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47739.log.gz 2024-04-15T15:26:43.647 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.c.log: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-15T15:26:43.647 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53552.log 2024-04-15T15:26:43.647 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53720.log.gz 2024-04-15T15:26:43.648 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53888.log 2024-04-15T15:26:43.648 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44210.log.gz 2024-04-15T15:26:43.649 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52671.log 2024-04-15T15:26:43.649 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53552.log.gz 2024-04-15T15:26:43.650 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42064.log 2024-04-15T15:26:43.650 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53888.log.gz 2024-04-15T15:26:43.651 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50890.log 2024-04-15T15:26:43.651 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52671.log.gz 2024-04-15T15:26:43.652 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42064.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42064.log.gz 2024-04-15T15:26:43.652 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46176.log 2024-04-15T15:26:43.653 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49774.log 2024-04-15T15:26:43.654 INFO:teuthology.orchestra.run.smithi150.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50890.log.gz 2024-04-15T15:26:43.654 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49094.log 2024-04-15T15:26:43.655 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46176.log.gz 2024-04-15T15:26:43.655 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42614.log 2024-04-15T15:26:43.656 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49774.log.gz 2024-04-15T15:26:43.656 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52503.log 2024-04-15T15:26:43.657 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49094.log.gz 2024-04-15T15:26:43.658 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43167.log 2024-04-15T15:26:43.658 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42614.log.gz 2024-04-15T15:26:43.658 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53109.log 2024-04-15T15:26:43.659 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52503.log.gz 2024-04-15T15:26:43.660 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43167.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-15T15:26:43.660 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43167.log.gz 2024-04-15T15:26:43.661 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50451.log 2024-04-15T15:26:43.661 INFO:teuthology.orchestra.run.smithi150.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53109.log.gz 2024-04-15T15:26:43.662 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50263.log 2024-04-15T15:26:43.663 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.0.50451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54266.log 2024-04-15T15:26:43.663 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50451.log.gz 2024-04-15T15:26:43.663 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-15T15:26:43.664 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-04-15T15:26:43.664 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50263.log.gz 2024-04-15T15:26:43.665 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54098.log 2024-04-15T15:26:43.665 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54266.log.gz 2024-04-15T15:26:43.666 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-04-15T15:26:43.666 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-04-15T15:26:43.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49700.log 2024-04-15T15:26:43.667 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54098.log.gz 2024-04-15T15:26:43.668 INFO:teuthology.orchestra.run.smithi187.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-15T15:26:43.668 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53194.log 2024-04-15T15:26:43.668 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-04-15T15:26:43.669 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41814.log 2024-04-15T15:26:43.669 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49700.log.gz 2024-04-15T15:26:43.670 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42362.log 2024-04-15T15:26:43.670 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53194.log.gz 2024-04-15T15:26:43.671 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48615.log 2024-04-15T15:26:43.671 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41814.log.gz 2024-04-15T15:26:43.672 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42530.log 2024-04-15T15:26:43.672 INFO:teuthology.orchestra.run.smithi150.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42362.log.gz 2024-04-15T15:26:43.673 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45283.log 2024-04-15T15:26:43.673 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48615.log.gz 2024-04-15T15:26:43.674 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47907.log 2024-04-15T15:26:43.675 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42530.log.gz 2024-04-15T15:26:43.675 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.2.45283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45283.log.gz 2024-04-15T15:26:43.675 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-04-15T15:26:43.676 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53594.log 2024-04-15T15:26:43.677 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-04-15T15:26:43.677 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50559.log 2024-04-15T15:26:43.678 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-04-15T15:26:43.679 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51667.log 2024-04-15T15:26:43.679 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53594.log.gz 2024-04-15T15:26:43.679 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53236.log 2024-04-15T15:26:43.680 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50559.log.gz 2024-04-15T15:26:43.680 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53362.log 2024-04-15T15:26:43.681 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51667.log.gz 2024-04-15T15:26:43.682 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51709.log 2024-04-15T15:26:43.682 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53236.log.gz 2024-04-15T15:26:43.683 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46539.log 2024-04-15T15:26:43.683 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53362.log.gz 2024-04-15T15:26:43.684 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43832.log 2024-04-15T15:26:43.684 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51709.log.gz 2024-04-15T15:26:43.685 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44000.log 2024-04-15T15:26:43.685 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46539.log.gz 2024-04-15T15:26:43.686 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43165.log 2024-04-15T15:26:43.686 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43832.log.gz 2024-04-15T15:26:43.687 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54140.log 2024-04-15T15:26:43.687 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44000.log.gz 2024-04-15T15:26:43.688 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52281.log 2024-04-15T15:26:43.688 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43165.log.gz 2024-04-15T15:26:43.689 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54140.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54140.log.gz /var/log/ceph/ceph-client.admin.53007.log 2024-04-15T15:26:43.689 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-15T15:26:43.690 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-04-15T15:26:43.691 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52281.log.gz 2024-04-15T15:26:43.691 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37622.log 2024-04-15T15:26:43.691 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53007.log.gz 2024-04-15T15:26:43.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49052.log 2024-04-15T15:26:43.692 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-04-15T15:26:43.693 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46008.log 2024-04-15T15:26:43.693 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37622.log.gz 2024-04-15T15:26:43.694 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44474.log 2024-04-15T15:26:43.694 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49052.log.gz 2024-04-15T15:26:43.695 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45924.log 2024-04-15T15:26:43.695 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46008.log.gz 2024-04-15T15:26:43.696 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42404.log 2024-04-15T15:26:43.696 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44474.log.gz 2024-04-15T15:26:43.697 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54504.log 2024-04-15T15:26:43.697 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45924.log.gz 2024-04-15T15:26:43.698 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47361.log 2024-04-15T15:26:43.699 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42404.log.gz 2024-04-15T15:26:43.699 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54504.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-15T15:26:43.700 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54504.log.gz 2024-04-15T15:26:43.700 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43510.log 2024-04-15T15:26:43.701 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47361.log.gz 2024-04-15T15:26:43.701 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50517.log 2024-04-15T15:26:43.703 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41980.log 2024-04-15T15:26:43.703 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43510.log.gz 2024-04-15T15:26:43.704 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53846.log 2024-04-15T15:26:43.704 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50517.log.gz 2024-04-15T15:26:43.705 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45966.log 2024-04-15T15:26:43.705 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41980.log.gz 2024-04-15T15:26:43.705 INFO:teuthology.orchestra.run.smithi187.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-15T15:26:43.706 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45504.log 2024-04-15T15:26:43.706 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53846.log.gz 2024-04-15T15:26:43.707 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46302.log 2024-04-15T15:26:43.707 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45966.log.gz 2024-04-15T15:26:43.709 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42837.log 2024-04-15T15:26:43.709 INFO:teuthology.orchestra.run.smithi150.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45504.log.gz 2024-04-15T15:26:43.709 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53804.log 2024-04-15T15:26:43.710 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46302.log.gz 2024-04-15T15:26:43.711 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43160.log 2024-04-15T15:26:43.711 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42837.log.gz 2024-04-15T15:26:43.712 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51997.log 2024-04-15T15:26:43.712 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53804.log.gz 2024-04-15T15:26:43.713 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48194.log 2024-04-15T15:26:43.713 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43160.log.gz 2024-04-15T15:26:43.714 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-04-15T15:26:43.715 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51997.log.gz 2024-04-15T15:26:43.716 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49616.log 2024-04-15T15:26:43.716 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48194.log.gz 2024-04-15T15:26:43.717 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52587.log 2024-04-15T15:26:43.717 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-04-15T15:26:43.718 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44126.log 2024-04-15T15:26:43.718 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49616.log.gz 2024-04-15T15:26:43.719 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42150.log 2024-04-15T15:26:43.720 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52587.log.gz 2024-04-15T15:26:43.720 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46869.log 2024-04-15T15:26:43.721 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44126.log.gz 2024-04-15T15:26:43.721 INFO:teuthology.orchestra.run.smithi187.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-15T15:26:43.722 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48113.log 2024-04-15T15:26:43.722 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42150.log.gz 2024-04-15T15:26:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2024-04-15T15:26:43.723 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46869.log.gz 2024-04-15T15:26:43.724 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46497.log 2024-04-15T15:26:43.724 INFO:teuthology.orchestra.run.smithi150.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz 2024-04-15T15:26:43.725 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52713.log 2024-04-15T15:26:43.725 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gz 2024-04-15T15:26:43.726 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48531.log 2024-04-15T15:26:43.727 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46497.log.gz 2024-04-15T15:26:43.728 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46827.log 2024-04-15T15:26:43.728 INFO:teuthology.orchestra.run.smithi150.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52713.log.gz 2024-04-15T15:26:43.729 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44846.log 2024-04-15T15:26:43.729 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48531.log.gz 2024-04-15T15:26:43.730 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53972.log 2024-04-15T15:26:43.730 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46827.log.gz 2024-04-15T15:26:43.731 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.0.44846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44846.log.gz 2024-04-15T15:26:43.731 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54182.log 2024-04-15T15:26:43.732 INFO:teuthology.orchestra.run.smithi187.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-15T15:26:43.733 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54464.log 2024-04-15T15:26:43.733 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53972.log.gz 2024-04-15T15:26:43.734 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54182.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.54182.log.gz --verbose 2024-04-15T15:26:43.734 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/ceph-client.admin.52039.log 2024-04-15T15:26:43.735 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48573.log 2024-04-15T15:26:43.735 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54464.log.gz 2024-04-15T15:26:43.736 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52039.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52039.log.gz 2024-04-15T15:26:43.736 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48279.log 2024-04-15T15:26:43.737 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47655.log 2024-04-15T15:26:43.737 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48573.log.gz 2024-04-15T15:26:43.738 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51460.log 2024-04-15T15:26:43.738 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48279.log.gz 2024-04-15T15:26:43.740 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43169.log 2024-04-15T15:26:43.740 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47655.log.gz 2024-04-15T15:26:43.741 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51460.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-15T15:26:43.741 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51460.log.gz 2024-04-15T15:26:43.742 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42488.log 2024-04-15T15:26:43.742 INFO:teuthology.orchestra.run.smithi150.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43169.log.gz 2024-04-15T15:26:43.743 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-04-15T15:26:43.744 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-15T15:26:43.745 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46092.log 2024-04-15T15:26:43.745 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42488.log.gz 2024-04-15T15:26:43.746 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45462.log 2024-04-15T15:26:43.746 INFO:teuthology.orchestra.run.smithi150.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-04-15T15:26:43.747 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54308.log 2024-04-15T15:26:43.747 INFO:teuthology.orchestra.run.smithi150.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46092.log.gz 2024-04-15T15:26:43.749 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52797.log 2024-04-15T15:26:43.749 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45462.log.gz 2024-04-15T15:26:43.750 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42022.log 2024-04-15T15:26:43.750 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54308.log.gz 2024-04-15T15:26:43.751 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54056.log 2024-04-15T15:26:43.752 INFO:teuthology.orchestra.run.smithi187.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-15T15:26:43.752 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52797.log.gz 2024-04-15T15:26:43.752 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54350.log 2024-04-15T15:26:43.753 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42022.log.gz 2024-04-15T15:26:43.754 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49816.log 2024-04-15T15:26:43.754 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54056.log.gz 2024-04-15T15:26:43.755 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54014.log 2024-04-15T15:26:43.755 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54350.log.gz 2024-04-15T15:26:43.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47277.log 2024-04-15T15:26:43.757 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49816.log.gz 2024-04-15T15:26:43.758 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48842.log 2024-04-15T15:26:43.758 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54014.log.gz 2024-04-15T15:26:43.759 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47138.log 2024-04-15T15:26:43.759 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47277.log.gz 2024-04-15T15:26:43.760 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53930.log 2024-04-15T15:26:43.761 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48842.log.gz 2024-04-15T15:26:43.762 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49658.log 2024-04-15T15:26:43.762 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47138.log.gz 2024-04-15T15:26:43.763 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45840.log 2024-04-15T15:26:43.763 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53930.log.gz 2024-04-15T15:26:43.764 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44294.log 2024-04-15T15:26:43.764 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49658.log.gz 2024-04-15T15:26:43.766 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2024-04-15T15:26:43.766 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45840.log.gz 2024-04-15T15:26:43.767 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44294.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-15T15:26:43.767 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44294.log.gz 2024-04-15T15:26:43.768 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50062.log 2024-04-15T15:26:43.768 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2024-04-15T15:26:43.769 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51018.log 2024-04-15T15:26:43.771 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47001.log 2024-04-15T15:26:43.771 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50062.log.gz 2024-04-15T15:26:43.772 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51835.log 2024-04-15T15:26:43.773 INFO:teuthology.orchestra.run.smithi150.stderr: 91.1%/var/log/ceph/ceph-client.admin.47001.log: -- replaced with /var/log/ceph/ceph-client.admin.51018.log.gz 2024-04-15T15:26:43.774 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41619.log 2024-04-15T15:26:43.774 INFO:teuthology.orchestra.run.smithi150.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47001.log.gz 2024-04-15T15:26:43.775 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49574.log 2024-04-15T15:26:43.775 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-15T15:26:43.775 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51835.log.gz 2024-04-15T15:26:43.776 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49858.log 2024-04-15T15:26:43.776 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41619.log.gz 2024-04-15T15:26:43.777 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42835.log 2024-04-15T15:26:43.777 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49574.log.gz 2024-04-15T15:26:43.779 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43509.log 2024-04-15T15:26:43.779 INFO:teuthology.orchestra.run.smithi150.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49858.log.gz 2024-04-15T15:26:43.780 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43498.log 2024-04-15T15:26:43.780 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42835.log.gz 2024-04-15T15:26:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45630.log 2024-04-15T15:26:43.781 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43498.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43509.log.gz 2024-04-15T15:26:43.782 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44168.log 2024-04-15T15:26:43.782 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43498.log.gz 2024-04-15T15:26:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45630.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45630.log.gz 2024-04-15T15:26:43.783 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49900.log 2024-04-15T15:26:43.784 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46623.log 2024-04-15T15:26:43.785 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44168.log.gz 2024-04-15T15:26:43.786 INFO:teuthology.orchestra.run.smithi187.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-15T15:26:43.786 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51955.log 2024-04-15T15:26:43.786 INFO:teuthology.orchestra.run.smithi150.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49900.log.gz 2024-04-15T15:26:43.787 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46623.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46623.log.gz 2024-04-15T15:26:43.787 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45546.log 2024-04-15T15:26:43.788 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54224.log 2024-04-15T15:26:43.789 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51955.log.gz 2024-04-15T15:26:43.789 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45882.log 2024-04-15T15:26:43.789 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45546.log.gz 2024-04-15T15:26:43.790 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-04-15T15:26:43.790 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m0.260s 2024-04-15T15:26:43.790 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m0.933s 2024-04-15T15:26:43.790 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m0.225s 2024-04-15T15:26:43.792 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.54224.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54224.log.gz 2024-04-15T15:26:43.792 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48703.log 2024-04-15T15:26:43.793 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44336.log 2024-04-15T15:26:43.793 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48703.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45882.log.gz 2024-04-15T15:26:43.793 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48703.log.gz 2024-04-15T15:26:43.793 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48236.log 2024-04-15T15:26:43.794 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42838.log 2024-04-15T15:26:43.795 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48236.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44336.log.gz 2024-04-15T15:26:43.795 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47823.log 2024-04-15T15:26:43.795 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48236.log.gz 2024-04-15T15:26:43.796 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52197.log 2024-04-15T15:26:43.797 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42838.log.gz 2024-04-15T15:26:43.797 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48659.log 2024-04-15T15:26:43.798 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47823.log.gz 2024-04-15T15:26:43.799 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52197.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-15T15:26:43.799 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52197.log.gz 2024-04-15T15:26:43.800 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48659.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48659.log.gz 2024-04-15T15:26:43.800 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41718.log 2024-04-15T15:26:43.801 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50146.log 2024-04-15T15:26:43.802 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44979.log 2024-04-15T15:26:43.803 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41718.log.gz 2024-04-15T15:26:43.803 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52545.log 2024-04-15T15:26:43.804 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50146.log.gz 2024-04-15T15:26:43.805 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43497.log 2024-04-15T15:26:43.805 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44979.log.gz 2024-04-15T15:26:43.806 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49514.log 2024-04-15T15:26:43.806 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52545.log.gz 2024-04-15T15:26:43.807 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53152.log 2024-04-15T15:26:43.807 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43497.log.gz 2024-04-15T15:26:43.809 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49514.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49514.log.gz -5 2024-04-15T15:26:43.809 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43958.log 2024-04-15T15:26:43.809 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52081.log 2024-04-15T15:26:43.810 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53152.log.gz 2024-04-15T15:26:43.810 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43958.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.43958.log.gz -5 2024-04-15T15:26:43.811 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/ceph.log 2024-04-15T15:26:43.811 INFO:teuthology.orchestra.run.smithi150.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-15T15:26:43.812 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.50747.log/var/log/ceph/ceph-client.admin.52081.log: 2024-04-15T15:26:43.812 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52081.log.gz 2024-04-15T15:26:43.813 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47089.log 2024-04-15T15:26:43.813 INFO:teuthology.orchestra.run.smithi150.stderr: 85.3%/var/log/ceph/ceph-client.2.50747.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-04-15T15:26:43.814 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.50747.log.gz 2024-04-15T15:26:43.814 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48447.log 2024-04-15T15:26:43.815 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43503.log 2024-04-15T15:26:43.815 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47089.log.gz 2024-04-15T15:26:43.816 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48447.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-15T15:26:43.816 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48447.log.gz 2024-04-15T15:26:43.817 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45287.log 2024-04-15T15:26:43.817 INFO:teuthology.orchestra.run.smithi150.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43503.log.gz 2024-04-15T15:26:43.818 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44252.log 2024-04-15T15:26:43.819 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.2.45287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45287.log.gz 2024-04-15T15:26:43.820 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44809.log 2024-04-15T15:26:43.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44252.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44252.log.gz 2024-04-15T15:26:43.821 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2024-04-15T15:26:43.822 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.0.44809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43790.log 2024-04-15T15:26:43.823 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44809.log.gz 2024-04-15T15:26:43.824 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48796.log 2024-04-15T15:26:43.824 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2024-04-15T15:26:43.825 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43790.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43790.log.gz 2024-04-15T15:26:43.825 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-15T15:26:43.826 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-04-15T15:26:43.827 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48796.log.gz 2024-04-15T15:26:43.827 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47529.log 2024-04-15T15:26:43.828 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42194.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-15T15:26:43.829 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52239.log 2024-04-15T15:26:43.829 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gz 2024-04-15T15:26:43.830 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50020.log 2024-04-15T15:26:43.830 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47529.log.gz 2024-04-15T15:26:43.831 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44084.log 2024-04-15T15:26:43.832 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52239.log.gz 2024-04-15T15:26:43.833 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50813.log 2024-04-15T15:26:43.833 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50020.log.gz 2024-04-15T15:26:43.834 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45588.log 2024-04-15T15:26:43.834 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44084.log.gz 2024-04-15T15:26:43.835 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.50813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2024-04-15T15:26:43.835 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50813.log.gz 2024-04-15T15:26:43.836 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41937.log 2024-04-15T15:26:43.836 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45588.log.gz 2024-04-15T15:26:43.837 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42826.log 2024-04-15T15:26:43.837 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2024-04-15T15:26:43.838 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49942.log 2024-04-15T15:26:43.839 INFO:teuthology.orchestra.run.smithi150.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41937.log.gz 2024-04-15T15:26:43.839 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47571.log 2024-04-15T15:26:43.840 INFO:teuthology.orchestra.run.smithi150.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42826.log.gz 2024-04-15T15:26:43.841 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52629.log 2024-04-15T15:26:43.841 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49942.log.gz 2024-04-15T15:26:43.842 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47319.log 2024-04-15T15:26:43.842 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47571.log.gz 2024-04-15T15:26:43.843 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42572.log 2024-04-15T15:26:43.843 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52629.log.gz 2024-04-15T15:26:43.845 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51751.log 2024-04-15T15:26:43.845 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47319.log.gz 2024-04-15T15:26:43.845 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42572.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42572.log.gz -5 2024-04-15T15:26:43.845 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47697.log 2024-04-15T15:26:43.846 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48754.log 2024-04-15T15:26:43.847 INFO:teuthology.orchestra.run.smithi150.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51751.log.gz 2024-04-15T15:26:43.848 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53636.log 2024-04-15T15:26:43.848 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47697.log.gz 2024-04-15T15:26:43.848 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.48754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47445.log 2024-04-15T15:26:43.849 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48754.log.gz 2024-04-15T15:26:43.850 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53678.log 2024-04-15T15:26:43.850 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53636.log.gz 2024-04-15T15:26:43.851 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.47445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42834.log 2024-04-15T15:26:43.852 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47445.log.gz 2024-04-15T15:26:43.853 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.53678.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.49244.log -- replaced with /var/log/ceph/ceph-client.admin.53678.log.gz 2024-04-15T15:26:43.853 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-15T15:26:43.854 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45756.log 2024-04-15T15:26:43.855 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42834.log.gz 2024-04-15T15:26:43.855 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42830.log 2024-04-15T15:26:43.855 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49244.log.gz 2024-04-15T15:26:43.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52965.log 2024-04-15T15:26:43.856 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45756.log.gz 2024-04-15T15:26:43.858 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43173.log 2024-04-15T15:26:43.858 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42830.log.gz 2024-04-15T15:26:43.858 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-15T15:26:43.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.52965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2024-04-15T15:26:43.859 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52965.log.gz 2024-04-15T15:26:43.860 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43173.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43173.log.gz 2024-04-15T15:26:43.860 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46665.log 2024-04-15T15:26:43.861 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.45420.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-15T15:26:43.861 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.46665.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2024-04-15T15:26:43.862 INFO:teuthology.orchestra.run.smithi150.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46665.log.gz 2024-04-15T15:26:43.862 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-04-15T15:26:43.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37366.log 2024-04-15T15:26:43.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51506.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-04-15T15:26:43.864 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49010.log 2024-04-15T15:26:43.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51548.log 2024-04-15T15:26:43.866 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.49010.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.37366.log.gz 2024-04-15T15:26:43.866 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-04-15T15:26:43.868 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.51548.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51548.log.gz 2024-04-15T15:26:43.913 INFO:teuthology.orchestra.run.smithi150.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-15T15:26:43.915 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-15T15:26:43.935 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-15T15:26:43.962 INFO:teuthology.orchestra.run.smithi150.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-15T15:26:43.966 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-15T15:26:43.966 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m0.440s 2024-04-15T15:26:43.967 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m0.951s 2024-04-15T15:26:43.967 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.790s 2024-04-15T15:26:43.968 INFO:tasks.ceph:Archiving logs... 2024-04-15T15:26:43.968 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/remote/smithi150/log 2024-04-15T15:26:43.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-15T15:26:44.402 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/remote/smithi187/log 2024-04-15T15:26:44.404 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-15T15:26:44.626 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-15T15:26:44.648 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-15T15:26:44.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-15T15:26:44.655 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-15T15:26:44.805 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-15T15:26:44.814 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-15T15:26:44.815 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:26:44.824 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-15T15:26:44.870 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:26:44.881 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-04-15T15:26:44.882 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:26:44.883 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-15T15:26:44.883 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 42 +1614us[+1614us] +/- 43ms 2024-04-15T15:26:44.883 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -1644us[-1651us] +/- 57ms 2024-04-15T15:26:44.883 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -634us[ -641us] +/- 81ms 2024-04-15T15:26:44.883 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:26:44.889 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-15T15:26:44.889 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-04-15T15:26:44.889 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 41 +1619us[+1619us] +/- 43ms 2024-04-15T15:26:44.889 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -1620us[-1620us] +/- 57ms 2024-04-15T15:26:44.889 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -625us[ -624us] +/- 81ms 2024-04-15T15:26:44.889 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-15T15:26:44.890 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-15T15:26:44.900 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-15T15:26:44.900 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-15T15:26:44.909 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-15T15:26:45.109 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-15T15:26:45.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:26:45.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:26:45.418 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-15T15:26:45.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:26:45.522 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-15T15:26:45.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-15T15:26:45.544 INFO:teuthology.task.internal:Duration was 739.069895 seconds 2024-04-15T15:26:45.544 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-15T15:26:45.554 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-15T15:26:45.554 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-15T15:26:45.562 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-15T15:26:45.721 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:26:45.723 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-15T15:26:46.225 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-15T15:26:46.225 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-04-15T15:26:46.226 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-15T15:26:46.288 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-04-15T15:26:46.289 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-15T15:26:46.365 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-15T15:26:46.365 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-15T15:26:46.373 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-15T15:26:46.466 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-15T15:26:46.466 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-15T15:26:46.472 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-15T15:26:46.649 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-15T15:26:46.659 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-15T15:26:46.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-15T15:26:46.753 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-15T15:26:46.850 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-15T15:26:46.861 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:26:46.869 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:26:46.973 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-04-15T15:26:46.975 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-04-15T15:26:47.030 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:26:47.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:26:47.078 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-15T15:26:47.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-15T15:26:47.134 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-15T15:26:47.145 INFO:teuthology.task.internal:Transferring archived files... 2024-04-15T15:26:47.147 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/remote/smithi150 2024-04-15T15:26:47.147 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-15T15:26:47.257 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136/remote/smithi187 2024-04-15T15:26:47.257 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-15T15:26:47.365 INFO:teuthology.task.internal:Removing archive directory... 2024-04-15T15:26:47.365 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-15T15:26:47.373 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-15T15:26:47.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-15T15:26:47.440 INFO:teuthology.task.internal:Not uploading archives. 2024-04-15T15:26:47.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-15T15:26:47.450 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-15T15:26:47.451 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-15T15:26:47.456 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-15T15:26:47.506 INFO:teuthology.orchestra.run.smithi150.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 15 15:26 /home/ubuntu/cephtest 2024-04-15T15:26:47.511 INFO:teuthology.orchestra.run.smithi187.stdout: 395194 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 15 15:26 /home/ubuntu/cephtest 2024-04-15T15:26:47.514 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-15T15:26:47.526 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-15T15:26:47.546 INFO:teuthology.nuke:Checking targets against current locks 2024-04-15T15:26:47.573 DEBUG:teuthology.nuke:shortname: smithi150 2024-04-15T15:26:47.574 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:26:47.598 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 15:04:05.993206', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFuSZ/DP/wyAzuGlxUJi84I4XzTKzK+ihI5+e8VoYfLqsqaJE6ZyFbAYK2v+KqUj7YOOSmmxA9V2zQuEE9dpV8='} 2024-04-15T15:26:47.601 DEBUG:teuthology.nuke:shortname: smithi187 2024-04-15T15:26:47.601 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-15T15:26:47.621 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-15_14:54:08-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657136', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-15 15:04:05.991228', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB4AzTXlZerE2v7/Ddv2PX3441QCJcjmCtHgGL+/aNwq5dX8jKtNz9IMUa1Wui1rXYF+npugXtLCPABO/UijQRk='} 2024-04-15T15:26:47.652 INFO:teuthology.orchestra.console.smithi150:Power off 2024-04-15T15:26:47.652 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-15T15:26:47.676 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2024-04-15T15:26:47.676 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:26:47.692 INFO:teuthology.orchestra.console.smithi187:Power off 2024-04-15T15:26:47.692 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-15T15:26:47.702 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-15T15:26:47.715 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2024-04-15T15:26:47.715 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:26:47.736 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-15T15:26:51.703 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:26:51.738 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:26:51.814 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-15T15:26:51.847 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-15T15:26:55.816 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:26:55.847 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:26:55.926 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-04-15T15:26:55.957 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-15T15:26:56.027 INFO:teuthology.orchestra.console.smithi150:Power off completed 2024-04-15T15:26:59.962 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-15T15:27:00.072 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-04-15T15:27:00.173 INFO:teuthology.orchestra.console.smithi187:Power off completed 2024-04-15T15:27:00.278 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 client.1-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} duration: 739.0698952674866 failure_reason: 'Test failure: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=bb02693e2f3542b282d209b8830efe97 status: fail success: false 2024-04-15T15:27:00.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-15T15:27:00.357 INFO:teuthology.run:FAIL